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ARMY  INSTITUTIONAL  TRAINING:  CURRENT  STATUS  AND  FUTURE  RESEARCH 
EXECUTIVE  SUMMARY 


Research  Requirement: 

The  U.S.  Army  Training  and  Doctrine  Command  (TRADOC)  and  the  U.S.  Army  Research 
Institute  for  the  Social  and  Behavioral  Sciences  (ARI)  conducted  a  “Science  of  Learning”  workshop 
(Quinkert,  et  al.)  in  2006.  One  purpose  of  the  workshop  was  to  address  the  availability  of 
information  on  “new  pedagogical  techniques,  procedures,  and  technologies.”  With  regards  to 
institutional  training,  the  workshop  recommended  the  Army  exploit  current  and  emerging  training 
and  learning  technology  and  begin  the  exploitation  of  various  learning  and  training  theoretical 
models.  Distributed  learning  (dL)  and  guided  experiential  learning  (GEL)  were  specifically 
mentioned  as  technology  and  modeling  examples  for  investigation. 

Procedure: 

The  ARI  Ft.  Benning  Research  Unit  conducted  this  overall  overview  of  previous,  extant,  and 
emerging  institutional  training  approaches  and  technologies  from  the  point-of-view  of  compatibility 
and  integrability  with  the  Army  environment.  The  overview  fell  into  two  general  areas:  a  critical 
review  of  currently  employed  military  institutional  education  and  training,  and  a  review  and  synthesis 
of  previous  and  current  training  relevant  research  and  technology. 

The  research  team  interviewed  trainers,  training  developers,  and  training  managers  at  seven 
Army  proponent  schools  with  regard  to  their  usage  of  current  training  technology,  with  an  emphasis 
on  dL.  The  team  also  conducted  a  review  of  selected  contemporary  topics  in  learning  technology  and 
in  learning  science.  The  objective  was  to  produce  the  current  report  as  a  “snapshot”  of  institutional 
training,  both  in  general  and  as  implemented  by  the  Army,  with  a  goal  of  subsequently  analyzing  the 
snapshot  to  glean  researchable  training  issues  from  it.  The  snapshot  was  taken  from  two  different 
perspectives:  a  survey  of  current  institutional  training  practices  and  training  issues  in  various  Army 
schools  and  a  review  and  a  synthesis  of  the  literature  covering  current  training  research  and 
technology  as  applicable  to  the  Army’s  institutional  training  environment. 

Findings: 

Findings  and  issues  are  categorized  as  “policy  issues”  and  “research  issues.”  Policy  issues, 
such  as  training  scheduling  and  availability  or  quality  of  training  technology,  are  presented  as  items 
with  relatively  straightforward,  direct  potential  solutions  that  can  be  analyzed  and  considered  for 
adoption  by  Army  institutional  training  management.  Research  issues,  such  as  modifying  training  to 
address  far  transfer  or  integrating  problem-centered  instructional  approaches  into  Army  training,  are 
presented  as  items  with  no  direct  solutions  and  that  are  suitable  for  further  investigation,  ranging 
from  basic  research  in  training  and  education  to  development  and  assessment  of  prototype  Army 
training  and  education  products. 

Utilization  and  Dissemination  of  Findings: 

Results  of  this  work  will  be  used  to  direct  research  in  institutional  training  under  ARI’s 
current  WP  360,  Blended  Learning  Solutions  for  Army  Training.  Various  TRADOC  elements, 
including  TRADOC  G  3/5/7  Training,  the  Army  Training  Support  Center  (ATSC),  various  Captains 
Career  Courses,  and  Noncommissioned  Officer  (NCO)  Academies  have  expressed  interest  in  both  the 
policy  and  research  issues,  and  in  any  research  products  that  may  result. 


v 


vi 


ARMY  INSTITUTIONAL  TRAINING:  CURRENT  STATUS  AND  FUTURE  RESEARCH 


CONTENTS 


Page 

INTRODUCTION . 1 

THE  EXTERNAL  ENVIRONMENT . 3 

Major  Training  Drivers . 3 

Soldier  Population . 3 

Training  Management  Considerations . 4 

Training  Effectiveness  Assessment . 5 

The  transfer  problem . 5 

Factors  influencing  transfer . 6 

Assessment  of  transfer  and  the  transfer  climate . 11 

Training  for  transfer . 15 

Practical  implications . 19 

THE  ARMY  INSTITUTIONAL  ENVIRONMENT . 19 

Training  Development  and  Design . 20 

Status  in  proponent  schools . 21 

Considerations  from  learning  science  and  technology . 35 

Instructor  Issues . 47 

Army  instructor  preparation . 47 

Army  instructor  characterization . 49 

DISCUSSION  AND  RECOMMENDATIONS . 50 

Policy  Issues . 50 

Soldier-related  issues . 50 

Course  relevance . 50 

Course  availability . 51 

Transfer  related  issues . 51 

dL-specific  issues . 5 1 

Research  Issues . 52 

The  Army  learner . 52 

Training  for  transfer . 52 

Pedagogical  issues . 53 

REFERENCES . 57 

APPENDIX  A.  dL  Institutional  Training  Interview  Questions . A-l 


vii 


CONTENTS  (continued) 


Page 

LIST  OF  TABLES 

TABLE  1.  LEARNING  TRAINING  SYSTEM  INVENTORY:  TRAINING  SPECIFIC 

SCALES . 12 

TABLE  2.  LEARNING  TRAINING  SYSTEM  INVENTORY:  GENERAL  SCALES . 13 

TABLE  3.  THE  TRANSFER  DISTANCE  CONCEPTUAL  MODEL  (HOLTON  & 

BALDWIN,  2003) . 15 

TABLE  4.  PROPERTIES  OF  TRADITIONAL  MEDIA  (ADAPTED  FROM  HOLDEN  & 

WESTFALL,  2005) . 29 

TABLE  5.  DESCRIPTION  OF  TRAINING  MEDIA  USED  FOR  dL . 30 

TABLE  6.  INSTRUCTIONAL  VALUE  AND  DEVELOPMENT  ARCHITECTURE  OF 

TRAINING  MEDIA  (ADAPTED  FROM  BERSIN  AND  ASSOCIATES,  2003)  ...31 

TABLE  7.  FEATURES  OF  AUTHORING  SOFTWARE . 34 

TABLE  8.  PRINCIPLES  OF  LEARNING  (ADAPTED  FROM  MERRILL,  2002) . 36 

TABLE  9.  CHARACTERIZATION  OF  MAYER’S  (1996)  LEARNING  “METAPHORS” . 38 

TABLE  10.  EXAMPLE  MEASUREMENTS  FOR  ASSESSMENT  METHOD  FORMATS 
FOR  KNOWLEDGE  LEVELS  FROM  GIJBELS  ET  AL.  (2005)  AND 
SUGRUE  (1995) . 46 

TABLE  11.  HIGH  LEVEL  CHARACTERIZATION  OF  ARMY  BASIC  INSTRUCTOR 

AND  SMALL  GROUP  INSTRUCTOR  TRAINING  COURSES’  CONTENT . 48 


viii 


ARMY  INSTITUTIONAL  TRAINING:  CURRENT  STATUS  AND  FUTURE 

RESEARCH 

Introduction 

The  U.S.  Army  divides  training  into  three  domains:  the  operational  domain,  the 
institutional  domain,  and  the  self  development  domain.  The  institutional  domain,  which  is  the 
concern  of  this  work,  comprises  the  initial  training,  professional  military  education  (PME),  and 
functional  training  delivered  at  training  base  centers  or  schools  (U.S.  Army,  2008).  Currently, 
and  for  the  foreseeable  future,  this  Anny  institutional  training  capability  faces  the  problem  of 
maintaining  (and,  in  some  cases,  increasing)  training  capacity  and  capability  while  experiencing 
a  marked  decrease  in  training  resources,  in  terms  of  training  time,  training  funds,  and  training 
personnel.  This  decrease  in  resources  is  occasioned  by  the  Army’s  re-allocating  funds  and 
personnel  to  address  the  current  operating  environment. 

In  recognition  of  and  in  anticipation  of  this  challenge,  the  U.S.  Army  Training  and 
Doctrine  Command  (TRADOC)  and  the  U.S.  Anny  Research  Institute  for  the  Social  and 
Behavioral  Sciences  (ARI)  conducted  a  “Science  of  Learning”  workshop  (Quinkert,  et  ah,  2007) 
in  2006.  One  purpose  of  the  workshop  was  to  address  the  availability  of  information  on  “new 
pedagogical  techniques,  procedures,  and  technologies”  that  might  be  exploited  to  meet  the 
challenge.  Part  of  the  output  of  the  workshop  was  a  set  of  recommendations  for  further 
investigation.  With  regards  to  institutional  training,  the  workshop  recommended  the  Anny 
exploit  current  and  emerging  training  and  learning  technology  and  begin  the  exploitation  of 
various  learning  and  training  theoretical  models.  Due  to  their  potential  for  increased  efficiencies 
in  instruction,  distributed  learning  (dL)  and  guided  experiential  learning  (GEL)  were  specifically 
mentioned  as  technology  and  modeling  examples  for  investigation. 

Partially  as  result  of  the  workshop  outcomes,  ARI  in  late  Fiscal  Year  2007  initiated  at  its 
Ft.  Benning  Research  Unit  a  program  of  research  in  Army  institutional  training.  One  of  the  first 
objectives  of  that  program  was  to  produce  the  current  report  as  a  “snapshot”  of  institutional 
training,  both  in  general  and  as  implemented  by  the  Army,  with  a  goal  of  subsequently  analyzing 
the  snapshot  to  glean  researchable  training  issues  from  it.  The  snapshot  was  to  be  taken  from 
two  different  perspectives:  (a)  a  survey  of  current  institutional  training  practices  and  training 
issues  in  various  Anny  schools  and  (b)  a  review  and  a  synthesis  of  the  literature  covering  current 
training  research  and  technology  as  applicable  to  the  Army’s  institutional  training  environment. 

The  two  different  perspectives  were  seen  as  necessary  because,  in  some  cases,  they  have 
the  potential  for  conflict.  It  was  quickly  realized  that  the  Army  institutional  training 
environment  has  the  potential  to  constrain  what  new  or  different  training  technologies  and 
techniques  might  be  adopted  and  then  also  to  constrain  how  those  technologies  and  techniques 
might  be  implemented.  Also,  even  though  a  training  technology  or  technique  may  be  found  to  be 
a  “fit”  within  the  Army  constraints,  it  cannot  in  general  be  immediately  and  capriciously 
“plugged  into”  the  on-going  institutional  training  environment.  Any  and  all  modifications  within 
that  environment  must  be  as  seamlessly  as  possible  transitioned  and  integrated  into  on-going 
training  (cf.  Vandergriff,  2006).  Thus,  there  is,  in  the  discussion  and  recommendations  section 


1 


to  this  report,  a  conscious  effort  to  temper  recommendations  for  implementation  and  for  research 
to  fit  within  the  constraints  of  the  Army  institutional  training  environment. 

With  these  two  perspectives  in  mind,  this  report  examines  the  overall  institutional 
training  question  in  terms  of  two  component  environments:  the  environment  external  to 
institutional  training  and  the  institutional  environment  which  exists  within  and  is  influenced  by 
the  external  environment. 

This  report  first  examines  the  external  environment.  The  enclosing  external  consists  of 
the  constraints,  requirements,  and  options  that  may  be  placed  on  the  institutional  training 
environment.  These  include  considerations  such  as  the  Army  Force  Generation  (ARFORGEN) 
cycle,  the  student-relevant  characteristics  of  the  Army  population,  the  training  objectives 
resulting  from  analytical  decomposition  of  mission  essential  task  lists,  training  system 
effectiveness  and  efficiency  assessments,  and  integration  with  TRADOC-level  training 
management  systems.  In  general,  any  “forcing”  function  which  the  institutional  environment 
does  not  directly  control  is  deemed  an  element  of  the  external  environment. 

The  second  section  addresses  the  institutional  environment.  It  consists  of  elements  such 
as  training  design,  training  development,  training  delivery,  instructor  preparation,  course 
assessment,  and  student  assessment.  As  would  be  expected,  the  major  part  of  this  section  is 
devoted  to  consideration  of  the  institutional  environment  and  consideration  of  the  applicable 
learning  science  and  technology.  To  gain  an  appreciation  of  current  Army  institutional  training, 
the  authors  visited  a  representative  number  of  proponent  “schoolhouses”  and  interviewed 
trainers,  training  developers,  and  training  managers  with  regard  to  local  current  training  practices 
(with  special  emphasis  on  dL),  to  local  emerging  training  practices,  and  to  perceived  institutional 
training  needs.  Special  effort  was  devoted  to  identifying  commonalities  across  training  entities. 
Also  in  this  section  appears  a  relatively  detailed  discussion  of  current  learning  science  and 
technology  from  the  perspective  of  Anny  institutional  training. 

Across  both  environments,  three  themes  are  given  special  attention:  applications  of 
learning  science,  methods  of  assessing  learning,  and  methods  of  assessing  transfer  of  learning. 
The  interest  in  applications  of  learning  science  arises,  of  course,  directly  from  the  overall 
objective  of  this  report.  But,  if  the  effectiveness  of  one  application  of  learning  science  is  to  be 
compared  to  that  of  another,  it  is  critical  that  there  be  some,  preferably  non-qualitative,  method 
of  measuring  and  assessing  the  effectiveness  of  both  applications.  Because  differences  in 
“effectiveness”  of  training  are  central  to  any  discussion  of  training  alternatives,  its  orientation 
toward  measurement  and  assessment  of  effectiveness  is  central  to  this  report.  Even  more  critical 
is  the  definition  of  “effectiveness”  -  is  the  effectiveness  of  a  course  to  train  a  given  subject 
domain  best  measured  by  Soldiers’  scores  on  an  end  of  course  test,  or  is  it  best  measured  by  their 
subsequent  performance  within  that  subject  domain?  Army  institutional  training  is  for  the  most 
part  applied  training;  that  is,  what  is  learned  is  to  be  applied  to  some  set  of  Army  tasks.  If  the 
Soldier  is  to  apply  the  training,  then  the  effectiveness  of  the  training  would  best  be  expressed  in 
tenns  of  its  eventual  application,  or  in  terms  of  how  well  the  training  transfers  to  the  operational 
context. 
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Of  these  three  topics,  applications  of  learning  science  and  assessment  of  transfer  are 
deemed  of  highest  importance  and  are  emphasized  in  the  following  sections. 

The  External  Environment 

The  external  environment  comprises  those  entities  outside  the  Anny  institutional 
environment  that  may  influence  and  are  influenced  by  the  institutional  environment.  Army- 
specific  areas  that  are  considered  below  include  conditions  or  drivers  that  are  imposed  from 
TRADOC  and  above,  characteristics  of  the  source  population  upon  which  the  Army  may  draw, 
TRADOC  training  management  policies,  requirements,  and  options,  and,  last,  considerations  for 
assessment  of  training  effectiveness. 

Major  Training  Drivers 

Army  Field  Manual  (FM)  7-0,  Training  for  Full  Spectrum  Operations  (US  Army,  2008) 
defines  the  general  external  environment  for  institutional  training.  Noting  that  “effective  training 
is  the  cornerstone  of  operational  success”  (p.  1-5),  FM  7-0  expects  Soldiers  to  be  “lifelong 
learners”  and  to  periodically  spiral  through  the  institutional  domain  to  acquire  skills  and 
knowledge  necessary  for  their  current  or  their  next  duty  assignment.  In  this  way,  institutional 
training  is  seen  as  a  resource  existing  to  support  the  operational  domain. 

In  supporting  the  operational  domain,  Soldiers’  institutional  training  must  operate  within 
the  ARFORGEN  process  (Department  of  the  Anny,  2007c).  Because  the  ARFORGEN  process, 
as  designed,  schedules  and  re-cycles  Brigade  Combat  Teams  through  a  3 -year,  3 -stage  “re¬ 
set/train  ->  ready  ->  available”  repeating  sequence,  TRADOC  can  in  principle  sync -up 
institutional  training  offerings  with  units’  ARFORGEN  schedules.  The  TRADOC  Commander 
has  listed  “Posture  TRADOC  to  support  ARFORGEN  implementation”  as  one  of  the  current 
TRADOC  Campaign  Plan’s  seven  primary  objectives  (Wallace,  2006).  In  a  few  cases, 
implementation  of  this  objective  has  resulted  in  tailoring  institutional  course  lengths  to  fit  the 
“re-set/train”  phase  and  also  in,  for  some  critical  courses,  modulating  throughput  rate  of  Soldier 
students  to  coordinate  with  units  coming  off  “available”  phase  deployment  (Wallace,  2006).  The 
expectation  is  that  TRADOC  will  continue  to  orient  institutional  training  toward  supporting 
ARFORGEN. 

Soldier  Population 

The  Soldier  population  that  institutional  training  supports,  being  of  age  18  years  or  older, 
would  be  classified  as  “adult  learners”  (Knowles  et  ah,  1998)  and  as  having  corresponding  adult 
learner  needs  and  expectations.  Various  writers  in  adult  education  (e.g.,  Knowles  et  al,  1998, 
Wlodkowski,  1993)  point  out  a  number  of  these  needs,  but,  most  important  for  Army 
institutional  training,  a  Soldier  adult  learner’s  training  must  be  perceived  as  directly  applicable  to 
the  Soldier’s  personal  situation.  The  Soldier’s  personal  environment  is  discussed  in  more  detail 
later  in  the  report,  but  one  overall  aspect  of  the  Soldier’s  personal  situation  is  experience  level, 
especially  academic  experience  level  the  Soldier  brings  with  her  or  him  on  entry  into  the  Army. 
Office  of  Army  Demographics  data  (Department  of  the  Army,  2007)  indicate  that  in  fiscal  year 
1995,  94%  of  non-prior  service  enlistees  had  earned  high  school  diplomas,  while,  in  fiscal  year 
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2006,  81%  of  non-prior  service  enlistees  had  earned  high  school  diplomas.  These  data  indicate 
that  different  year-groups  of  Soldiers  can  differ  in  their  expected  academic  baseline. 

Specifically,  in  2009,  Soldiers  with  approximately  14  years  service  (1995  accessions)  on  average 
have  a  public  school  background  different  from  Soldiers  with  approximately  3  years  service 
(2006  accessions).  To  a  certain  degree,  Army  institutional  training  must  take  into  consideration 
not  only  changes  in  course  content  in  response  to  the  current  operational  environment,  but  also 
changes  over  time  in  the  Soldier  population  that  is  served  by  the  courses. 

Although  there  is  the  possibility  that  cultural  differences  in  the  Soldier  population  could 
interact  with  institutional  instruction  (cf.  Hofstede,  1986),  the  Army  seeks  to  inculcate  in  its 
Soldiers  a  common  culture  of  “lifelong  learning”  (U.S.  Army  Training  and  Doctrine  Command, 
2006).  This  specific  acculturation  begins  at  Soldier  accession  and  continues  through  the 
Soldier’s  career.  As  indicated  above,  the  Army  expects  Soldiers  to  continue  to  train  throughout 
their  careers,  and  institutional  training  is  expected  to  provide  a  large  part  of  that  training. 
Soldiers,  on  the  other  hand,  expect  institutional  training  to  be  readily  available  and  to  be  of 
benefit  to  their  careers. 

Given  the  current  tempo  of  operations,  availability  and  applicability  of  institutional 
training  become  of  personal  importance  to  deploying  and  returning  Soldiers.  With  the 
ARFORGEN  cycle  force-fitted  to  the  current  tempo,  Soldiers  have  a  limited  available  period  of 
time  during  which  they  are  not  deployed,  and  they  feel  they  must  wring  maximum  benefit  from 
this  time.  Thus,  from  the  Soldier’s  point  of  view,  institutional  training  course  lengths  ideally  are 
at  most  a  few  weeks  in  length  and  course  content  is  ideally  concise  and  relevant.  The  Soldier’s 
expectation  of  “relevance”  is  that  the  course  be  up  to  date  to  reflect  current  operations  and  that 
the  course  content  level  be  appropriate.  Typical  Soldier  critical  comments  voice  dissatisfaction 
that  some  courses  appear  not  to  have  been  updated  for  years  and  that  some  courses  “try  to  teach 
me  the  job  I’ve  already  been  doing  for  the  past  six  months”  (M.  Quirion,  personal 
communication.  May  4,  2007).  TRADOC  Pamphlet  350-70-10  (U.S.  Army  Training  and 
Doctrine  Command,  2004)  emphasizes  course  content  validation,  with  the  unintended  result  that 
courses  are  not  consistently  validated  against  external  criteria. 

Training  Management  Considerations 

Army  Regulation  350-1  (Department  of  the  Army,  2007b)  outlines  the  Anny  institutional 
training  system.  By  policy,  institutional  training  is  managed  at  the  course  level;  in 
implementation,  all  institutional  courses  Army-wide  are  listed  in  and  scheduled  via  the  Army 
Training  Requirements  and  Resources  System  (ATRRS).  Course  offerings  and  schedules  are 
available  via  ATRRS,  and  prospective  students  almost  with  no  exception  schedule  their 
attendance  in  a  specific  course  at  a  specific  location  with  specific  course-start  and  course-end 
dates  via  ATRRS. 

One  of  the  unintended  effects  of  managing  training  at  the  course  level  is  the  resultant 
difficulties  in  personalizing  course  delivery.  For  a  specific  course,  Students  first  must  confonn 
their  personal  schedules  to  a  course  start  schedule,  and,  then,  once  in  attendance,  must,  in  most 
cases,  conform  to  the  course  end  schedule.  The  fixed  course  end  schedule  constrains  to  a  large 
extent  the  design  of  institutional  courses:  a  true  self-paced  course  or  a  course  with  a  separate 
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accelerated  “fast  track”  presents  administrative  difficulties  in  managing  how  to  accommodate  the 
“left  over”  time  of  Soldiers  who  complete  the  course  before  its  scheduled  end.  This  problem  is 
more  acute  in  those  cases  in  which  a  Soldier  may  be  scheduled  for  two  “back-to-back” 
institutional  courses  -  if  the  Soldier  completes  the  first  course  a  few  days  early  and  the  second 
course’s  start  date  is  set  at  a  fixed  date,  the  institution  is  faced  with  the  challenge  of  putting  the 
Soldier’s  between-courses  time  to  constructive  use.  The  expectation  is  that  TRADOC  will 
continue,  at  least  for  the  near  future,  to  manage  access  to  institutional  training  at  the  course  level. 

Training  Effectiveness  Assessment 

Army  training  and  education  has  the  purpose  of  supporting  the  Anny’s  mission  of 
providing  sustained  land  dominance  across  the  full  spectrum  of  conflict.  Implied  in  meeting  this 
purpose  is  the  ability  to  assess  the  training’s  effectiveness  in  supporting  the  mission.  This 
assessment  happens  outside  the  institutional  environment.  Measurement  and  assessment  of 
knowledge  acquired  during  training  is  admittedly  important,  but  more  critical  is  the  measurement 
and  assessment  of  the  real-world  application  of  that  knowledge  external  to  the  institutional 
environment.  So,  for  purposes  of  this  paper,  the  Army’s  assessment  of  training  effectiveness 
should  aim  at  determining  how  much  better  the  Soldier  performs  operationally  (on  the  job  or  in 
the  field)  subsequent  to  institutional  training.  This  subsequent,  external  performance 
improvement  is  termed  transfer  of  learning  or  transfer  of  training. 

Training  transfer  generally  refers  to  the  application  of  trained  knowledge  and  skills  back 
to  the  job  (Burke  &  Hutchins,  2007).  According  to  Baldwin  and  Ford  (1988),  two  conditions 
must  be  met  for  transfer  to  have  occurred:  1)  learned  behavior  must  be  generalized  to  the  job 
context,  and  2)  it  must  be  maintained  over  a  period  of  time  on  the  job.  Smith,  Ford,  and 
Kozlowski  (1997)  argued  for  a  third  condition  or  indicator  of  transfer:  the  extent  to  which  the 
trainee  can  adapt  to  novel  or  changing  situational  demands,  with  its  emphasis  on  developing 
effective  problem  solving  skills. 

The  transfer  problem.  Facteau,  Dobbins,  Russell,  Ladd,  and  Kudisch  (1995)  reported 
findings  that  organizations  spend  as  much  as  $200  billion  annually  on  workforce  training.  These 
costs  will  likely  increase  as  the  economy  continues  to  expand  in  the  high  technology,  service  and 
information  sectors.  Rapid  technological  development  will  place  greater  demands  on  workers  to 
improve  the  quality  of  services  and  products  due  to  increased  global  competition.  One  result  of 
this  economic  expansion  will  be  the  increasing  need  for  organizations  to  better  utilize  the 
available  workforce.  To  stay  competitive,  organizations  will  need  to  enhance  their  training 
activities  for  current  employees. 

Recent  studies,  however,  indicate  that  the  impact  of  training  programs  has  not  been  very 
effective.  Saks  (2002)  reports  survey  findings  indicating  about  40%  of  trainees  fail  to  show 
transfer  immediately  after  training,  70%  falter  in  transfer  one  year  after  the  program,  and 
ultimately  only  50%  of  training  investments  result  in  organizational  or  individual  improvements. 
Transfer  of  learning  continues  to  represent  a  core  issue  for  human  resource  researchers  and 
trainers  focused  on  designing  interventions  to  enhance  individual,  group,  and  organizational 
performance  (Yamnill  &  McLean,  2001). 
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Issues  related  to  transfer  and  transfer  effectiveness  can  be  grouped  in  three  categories: 

•  Factors  influencing  transfer 

•  Assessment  of  transfer  and  transfer  climate 

•  Training  for  transfer 

These  areas  will  be  briefly  discussed  and  key  findings  presented. 

Factors  influencing  transfer.  Since  1988,  three  comprehensive  reviews  have  been 
performed  (Baldwin  &  Ford,  1988;  Ford  &  Weissbein,  1997;  &  Burke  &  Hutchins,  2007) 
concerning  the  factors  influencing  transfer.  The  reviews  have  focused,  in  varying  degrees,  on 
three  primary  factors  -  learner  characteristics,  intervention  design  and  delivery,  and  work 
environment  influences. 

Learner  characteristics.  A  sizeable  body  of  work  has  sought  to  relate  degree  or  quality 
of  transfer  as  a  function  of  the  individual  characteristics  of  the  learner. 

Cognitive  ability.  General  mental  ability  has  been  extensively  studied  and  shown  to  be  a 
consistently  reliable  predictor  of  job  and  training  performance.  More  specifically,  far  transfer 
(the  application  of  learning  to  situations  dissimilar  to  those  of  the  original  learning  events, 
Yamnill  &  McLean,  [2001])  is  generally  achieved  by  students  with  higher  general  ability  scores 
(Clark  &  Voogel,  1985).  Extensive  meta-analyses  based  on  20  years  of  training  research  which 
examined  the  relationship  between  cognitive  ability  and  training  transfer  yielded  a  correlation  of 
.43  (Colquitt,  Le  Pine,  &  Noe,  2000).  While  the  findings  reported  in  this  area  have  been 
impressive,  cognitive  ability  is  fairly  resistant  to  training  intervention  strategies  and  its  value 
may  be  limited  primarily  as  a  co  variate  for  examining  the  impact  of  training  interventions  in 
groups  of  differing  ability  levels. 

Self-efficacy.  Self-efficacy  (competency  based  judgment  by  trainees  to  perform  specific 
tasks)  has  been  found  to  be  positively  related  to  transfer  generalization  and  transfer  maintenance 
across  multiple  studies  (e.g.,  Chiaburu  &  Marinova,  2005;  Gaudine  &  Saks,  2004,  cited  in  Burke 
&  Hutchins,  2007). 

Some  interventions  designed  to  increase  learner  self-efficacy  have  produced  increases  in 
training  performance.  For  example,  including  mastery  experiences  and  supportive  feedback 
(Gist,  1986)  or  goal  setting  and  self-maintenance-meta  cognitive  strategies  (Gist,  Stevens,  & 
Bavetta,  1991)  as  part  of  post  training  interventions  resulted  in  enhanced  transfer.  The  findings 
suggest  that  unlike  cognitive  ability,  self-efficacy  is  a  malleable  learner  quality  that  should  be 
considered  in  developing  transfer  intervention  strategies. 

Motivation.  Motivation,  or  more  specifically,  training  motivation,  as  defined  by 
Tannenbaum  and  Yukl  (1992),  refers  to  the  intensity  and  persistence  of  efforts  that  trainees  apply 
in  learning-oriented  improvement  activities,  before,  during,  and  after  training.  Two  types  of  goal 
orientation  have  been  identified.  The  first,  mastery  orientation,  refers  to  the  belief  that  effort 
leads  to  improved  training  outcomes.  Individuals  with  a  mastery  orientation  are  focused  on 
developing  new  skills,  understanding  learning  tasks  and  identifying  the  most 
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appropriate/optimum  learning  strategies.  Performance  orientation,  in  contrast,  refers  to  the  belief 
that  ability  is  demonstrated  by  perfonning  better  than  others.  The  focus  of  these  individuals  is  to 
strive  to  do  well  on  a  task  regardless  of  whether  they  have  acquired  all  the  skills  to  other  settings 
(Ford  &  Weissbein,  1997).  The  challenge  for  trainers  according  to  Ford  and  Weissbein  (1997)  is 
to  develop  interventions  that  support  a  mastery  orientation  and,  thus,  enhance  the  potential  for 
transfer  to  the  job  setting.  As  noted  by  Noe  (1986)  and  the  present  authors,  trainee/Soldier 
motivation  is  critical  if  the  new  training  interventions  are  to  have  any  impact.  Key  factors 
impacting  motivation  will  be  described  in  the  next  section  under  design  issues. 

Personality.  Increased  focus  of  the  role  played  by  motivational  factors  in  training 
transfer  has  advanced  researchers’/trainers’  understanding  of  how  motivation  impacts  the 
learning  process.  Much  less  attention  has  been  directed  to  other  trainee  characteristics  such  as 
personality  factors. 

Personality  variables  that  have  been  studied  in  relation  to  their  impact  on  various 
components  of  trainee  motivation  include  anxiety,  openness  to  experience,  sociability,  and 
conscientiousness  (see  Burke  and  Hutchins,  2007).  Particular  attention  has  been  given  to  the 
“big  five”  personality  factors  and  their  possible  roles  in  predicting  or  moderating  future 
performance  on  the  job.  These  factors  include  the  three  factors  mentioned  earlier,  openness  to 
experience,  conscientiousness,  agreeableness  (sociability)  in  addition  to  extraversion,  and 
emotional  stability.  Ford  and  Weissbein  (1997)  hypothesize  that  personality  factors  might  not 
only  be  predictive  of  future  job  performance  but  also  may  impact  an  individual’s  motivation  to 
leam,  learning  strategies  used  during  training,  skill  acquisition  rates,  and  training  transfer. 

The  variable  which  may  have  the  most  impact  on  training  transfer  and  is  most  malleable 
to  intervention  efforts  is  anxiety.  The  Colquitt  et  al.  (2000)  training  meta  analysis  found  anxiety 
produced  negative  correlations  with  every  training  outcome  in  their  sample  (e.g.,  motivation  to 
leam,  post  training  self-efficacy,  declarative  knowledge)  including  transfer.  Anxiety  has  been 
linked  to  reduced  training  motivation  which  results  in  less  transfer  (or  lower  intentions  to 
transfer).  Less  anxious  individuals  may  be  able  to  focus  on  the  training  tasks,  and  are  less 
distracted,  which  could  lead  to  higher  transfer.  Interventions  designed  to  mitigate  anxiety  in 
trainees  by  enhancing  confidence  and  improving  individual  levels  of  self-efficacy  through  some 
of  the  strategies  mentioned  earlier  may  warrant  further  consideration  by  trainers. 

Perceived  utility.  While  listed  as  a  learner  characteristic  by  Burke  and  Hutchins  (2007), 
this  factor  is  closely  linked  to  motivation  and  the  design  factor,  content  relevance.  To  achieve 
maximal  transfer,  learners  must  perceive  that  the  new  knowledge  and  skills  will  improve  a 
relevant  aspect  of  their  work  perfonnance  (Baldwin  &  Ford,  1988).  Increased  utility  will  impact 
motivation  to  leam  which  should  impact  both  learning  and  transfer. 

Career/job  variables.  Another  set  of  learner  characteristics  impacting  transfer  are  job 
and  career  variables.  Transfer  has  been  found  to  be  positively  influenced  by  trainees’  job 
involvement  (Mathieu,  Tannenbaum,  &  Salas,  1992)  which  refers  to  the  degree  to  which  an 
individual  identifies  with  the  job,  actively  participates  in  it,  and  considers  job  performance  as 
critical  to  his  or  her  self-worth.  Similarly,  organizational  commitment  (Colquitt  et  al.,  2000)  has 
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been  shown  to  produce  a  more  interested  learner  who  wants  to  gain  and  use  the  new  knowledge 
at  work. 


While  job  involvement  and  organizational  commitment  can  impact  learning  and  transfer, 
interventions  designed  to  enhance  Soldiers’  perceptions  in  these  areas  may  be  beyond  the  scope 
of  the  current  research.  It  is  possible,  however,  that  interventions  designed  to  improve  perceived 
utility  of  the  training  may  have  indirect  effects  on  these  variables. 

Intervention  design  and  delivery.  Perhaps  the  most  commonly  addressed  factor  in  the 
investigation  of  transfer  is  the  training  intervention  itself:  how  the  training  is  designed,  what  it 
consists  of,  and  how  it  is  delivered. 

Needs  analysis.  Effective  training  design/intervention  addresses  knowledge,  skill,  and 
ability  deficits  (Burke  &  Hutchins,  2007).  Through  established,  formalized  procedures  such  as 
instructional  system  design,  or  various  survey  feedback  mechanisms  which  funnel  information 
from  the  user  to  course  developers,  some  mechanism  is  required  to  insure  that  the  training 
provided  Soldiers/trainees  is  timely  and  relevant.  Inappropriate  content,  poor  or  nonexistent 
organizational  support,  and  inadequate  resources  all  impact  Soldier/trainee  perceptions  of  the 
relevance  of  the  training  received  and  the  importance  of  demonstrating  newly  acquired  behaviors 
and  skills  on  the  job  or  in  operational  units. 

Transfer  climate.  In  addition  to  conducting  needs  analyses,  trainers  must  also  look  at 
work  environment  factors  such  as  transfer  climate  (Holton,  Bates,  &  Ruona,  2000).  Transfer 
climate  is  seen  by  Holton  et  al.  as  a  mediating  variable  in  the  relationship  between 
organization/unit  environment  and  individual’s  job  attitudes  and  work  behavior.  Thus,  even 
when  learning  occurs  in  training,  the  transfer  climate  can  either  support  or  inhibit  application  of 
learning  on  the  job.  These  factors  must  also  be  assessed  to  provide  the  trainer  with  a  complete 
picture  of  the  transfer  environment  and  whether  other  issues  in  the  work  environment  must  be 
addressed  before  embarking  on  a  new  training  intervention.  Specific  factors  are  discussed  in  the 
section  on  work  environment  influences  in  the  next  main  section. 

Learning  goals.  Assuming  the  needs  analysis  indicates  that  a  training/learning 
intervention  is  needed  and  that  the  work  climate  is  supportive  of  the  intervention,  providing 
advance  organizers  in  the  form  of  clearly  communicated  performance  objectives,  conditions 
under  which  the  perfonnance  will  be  expected  to  occur  on  the  job,  and  criteria  for  acceptable 
performance  can  enhance  transfer  outcomes  (Kraiger,  Salas,  &  Cannon-Bowers,  1995).  This 
suggests  that  trainees  are  more  likely  to  exhibit  transfer  when  they  have  a  clear  understanding  of 
what  knowledge  and  behaviors  are  required  and  when  they  are  expected  to  demonstrate  these 
actions. 


Content  relevance.  For  near  transfer  to  occur  (improved  performance  in  one’s  current 
job),  the  more  the  training  content  and  program  reflect  the  actual  work  setting,  the  more 
successful  the  near  transfer  (Baldwin  &  Ford,  1988).  Burke  and  Hutchens  (2007)  report 
empirical  studies  showing  that  the  content  validity  of  the  training  module  was  significantly 
correlated  to  transfer  immediately  after  and  at  the  one  month  mark  following  training.  The 
findings  reported  indicate  that  trainees  must  see  a  close  relationship  between  training  content  and 
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work  tasks  to  optimize  transfer  (immediate  and  sustained)  in  the  work  setting.  A  key  implication 
from  this  research  is  that  a  thorough  needs  analysis  and  assessment  of  work  environment  factors 
must  be  performed  to  ensure  that  the  new  training  is  needed  and  will  be  viewed  as  relevant  (by 
the  trainee)  for  improving  job  performance. 

Instructional  strategies  and  methods.  Key  instructional  strategies  for  improving  transfer 
include  practice,  feedback,  behavioral  modeling,  and  self-management-metacognitition.  These 
areas  are  discussed  in  the  following  paragraphs. 

As  noted  by  Bransford,  Brown,  and  Cooking  (2000),  without  an  adequate  level  of  initial 
learning,  transfer  cannot  be  expected,  despite  the  relevance  of  the  training  to  the  individual.  The 
work  reported  by  Bransford  et  al.  highlights  the  importance  of  establishing  acceptable 
performance  criteria  that  must  be  met  by  trainees  to  successfully  meet  course  requirements  to 
maximize  transfer  effects.  Overlearning  may  be  required  (practice  even  after  correct 
performance  has  been  demonstrated)  for  skills  not  required  on  a  regular  basis  (e.g.,  Fisk, 

Hertzog,  Lee,  Rogers,  &  Anderson,  1994).  Learning  complex  tasks  requires  extensive  practice 
(Ericsson,  Krampe,  &  Tesch-Roemer  (1993).  Trainees  must  be  given  sufficient  time  and  varied 
experiences,  as  well  as  time  to  process  information,  to  enhance  transfer. 

The  type  of  learning  emphasized  during  training  has  a  direct  impact  on  subsequent 
transfer.  Memorizing  a  set  of  facts  or  procedures  may  not  be  conducive  to  transfer,  particularly 
far  transfer,  without  some  time  set  aside  for  insuring  deep  understanding  of  the  material 
presented  during  training.  This  requires  establishing  more  active  learning  protocols  involving 
increased  emphasis  on  problem  solving  exercises,  small  group  discussion,  and  feedback.  (See 
Burke  &  Hutchins,  2007;  Bransford,  Brown,  &  Cocking,  2000). 

Opportunities  for  feedback  should  occur  continuously,  as  part  of  the  instructional  process 
by  focusing  on  more  formative,  process  oriented  feedback  during  training  which  the  trainees  can 
use  to  revise  their  thinking  as  they  are  working  on  a  problem.  This  is  in  contrast  to  providing 
trainees  with  primarily  summative  feedback  in  the  form  of  test  scores  emphasizing  the  number  of 
facts  or  procedures  correctly  recalled.  The  emphasis  on  formative  feedback  is  designed  to 
enhance  deep  understanding  of  the  material  and  promote  transfer  (Bransford,  Brown,  &  Cocking, 
2000). 


Behavioral  modeling  provides  another  potential  transfer  strategy  by  enhancing  trainee 
self-efficacy  (Bandura,  1997).  Inclusion  of  different  situations  and/or  levels  of  model 
effectiveness  have  been  found  to  improve  trainee  retention  and  generalization  of  learned  skills 
for  higher  level  cognitive  functions  such  as  concept  fonnation  and  problem  solving,  as  well  as 
interpersonal  skills  (e.g.,  Baldwin  &  Ford,  1988;  Baldwin  cited  in  Baldwin  &  Ford,  1988).  In  a 
meta-analysis  of  1 17  studies,  Taylor,  Russ-Eft,  and  Chan  (2005)  found  that  the  use  of  mixed 
(positive  and  negative)  models  produced  greater  transfer  of  training  (changes  in  job  behavior) 
than  did  only  positive  models. 

Using  a  somewhat  related  instructional  strategy,  Smith-Jentsch,  Jentsch,  Payne,  and  Salas 
(1996)  employed  error-based  examples,  showing  trainees  what  could  go  wrong  if  they  did  not 
use  trained  skills  back  on  the  job.  Smith  et  al.  (1996)  found  that  this  strategy  promoted  transfer 
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in  experienced  trainees  (pilots)  who  had  encountered  specific  negative  events  related  to  the 
training.  The  effectiveness  of  this  approach  was  attributed  to  the  perceived  value 
(instrumentality)  of  the  training. 

Other  research  (see  Ivancic  &  Hesketh,  1996)  has  indicated  that  learning  environments 
designed  to  be  error-filled  experiences  can  be  quite  effective  for  improving  both  learning  and 
transfer.  The  effectiveness  of  this  approach  appears  to  come  from  enhanced  vigilance  in  the 
trainees’  produced  by  errors  which  results  in  a  clearer  mental  model  of  the  task  to  be  learned. 

Transfer  can  be  improved  by  helping  trainees  become  more  aware  of  themselves  as 
learners  who  actively  monitor  their  learning  strategies  and  resources  and  assess  their  readiness 
for  specific  tests  and  performances  (Bransford,  Brown,  &  Cocking,  2000).  Strategies 
emphasizing  self-  regulatory/management  behaviors  (e.g.,  Frayne  &  Latham,  1987)  and  goal 
setting  (e.g.,  Wexley  &  Baldwin,  1986)  have  been  shown  to  improve  transfer.  Wexley  and 
Baldwin  reported  enhanced  positive  transfer  two  months  after  receiving  the  original  (goal 
setting)  training. 

Work  and  environment  influences.  The  last  two  decades  have  seen  an  increased 
interest  in  this  research  area.  Much  of  the  previous  research  performed  has  focused  on  design 
factors.  Relatively  little  work  has  been  done  to  understand  how  work  environment  factors 
influence  transfer  of  training.  Organizations  interested  in  increasing  their  return  on  investment 
from  learning-training  interventions  must  develop  a  clearer  understanding  of  all  the  factors 
inhibiting  transfer  (Holton,  Bates,  &  Ruona,  2000).  Key  factors  impacting  the  overall  transfer 
climate  of  the  work  environment  are  briefly  summarized  below. 

Supervisor/peer  support.  The  role  of  supervisors  in  influencing  and  supporting  trainee 
transfer  has  been  widely  supported  in  both  empirical  and  qualitative  studies  (e.g.,  Clarke,  2002). 
Key  behaviors  engaged  in  by  managers  include:  discussing  new  learning  with  trainees, 
participating  in  training,  providing  encouragement  (positive  feedback)  and  coaching  to  trainees 
about  the  use  of  new  knowledge  and  skill(s)  on  the  job.  These  factors  were  most  identified  by 
trainees  as  positively  influencing  transfer  of  learning  (see  Burke  and  Hutchins  [2007]  for  a 
thorough  review). 

Peer/colleague  support.  Peer  and  colleague  support  has  been  shown  to  be  a  more 
consistent  influence  on  trainee  transfer  than  supervisory  support  (Facteau,  Dobbins,  Russell, 
Ladd  &  Kudisch,  1995).  Hawley  and  Barnard  (2005)  found  that  the  most  influential  peer 
support  behaviors  on  transfer  were  peer  networking  and  sharing  ideas  about  course  content. 
These  factors  helped  promote  skill  transfer  six  months  after  training. 

Opportunity  to  perform.  Limited  opportunities  to  perform  new  skills  back  on  the  job  was 
found  to  be  the  biggest  training  impediment  to  successful  training  transfer  (Clarke,  2002).  To 
insure  long  term  transfer  of  training  to  the  job,  trainees  must  be  provided  the  opportunity  to  use 
their  learned  skills.  Supervisors/leaders  play  a  key  role  in  creating  this  opportunity  (Axtel, 
Maitlis,  &  Yearta  (1997).  If  possible,  leaders  should  relook  their  subordinates’  workload  to 
allow  them  to  practice  new  skills  on  the  job  (e.g.,  Clarke,  2002). 
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Assessment  of  transfer  and  the  transfer  climate.  Transfer  of  learning  is  a  complex 
process  and  involves  multiple  factors  and  influences.  The  transfer  climate  may  either  support  or 
inhibit  application  of  learning  on  the  job  and  is  viewed  by  some  (e.g.,  Rouiller  &  Goldstein, 

1993)  as  at  least  as  important  as  learning  in  facilitating  transfer.  From  a  training  investment 
perspective,  it  is  important  that  the  trainer  or  practitioner  have  the  tools  to  accurately  diagnose 
those  factors  inhibiting  the  transfer  process  and  then  intervene  where  appropriate  (Holton,  Bates, 
&  Ruona,  2000). 

Kirkpatrick’s  four  level  evaluation  model.  Evaluation  of  interventions  is  one  of  the  most 
critical  issues  faced  by  training  practitioners  in  the  area  of  human  resource  development. 
However,  as  Holton  (1996)  points  out,  the  dominant  evaluation  model  currently  in  use 
(Kirkpatrick,  1998)  has  several  flaws  that  may  severely  limit  its  value  in  providing  trainers  with 
insights  for  improving  ineffective  training  interventions.  Kirkpatrick’s  (1998)  evaluation  model 
consists  of  four  levels  (reaction,  learning,  behavior,  and  results)  which  are  performed  in 
sequence,  from  trainee  reaction  to  results.  Briefly,  reaction  refers  to  how  the  trainees  react  to  the 
training  e.g.,  overall  like/dislike,  instructor  knowledge,  course  organization.  Learning  is  the 
extent  to  which  trainees  change  attitudes,  improve  knowledge,  and  or  increase  skill  as  a  result  of 
attending  the  program.  Behavior  is  defined  as  the  extent  to  which  change  in  behavior  (transfer  of 
learning  to  the  work  place)  has  occurred  because  the  participant  attended  the  training  program. 
Results  refer  to  the  final  objectives  of  the  program,  e.g.,  increased  production,  improved  quality, 
decreased  costs,  reduced  frequency  and/or  severity  of  accidents,  increased  sales,  reduced 
turnover,  higher  profits.  As  one  moves  from  one  level  to  the  next,  the  evaluation  process 
becomes  more  difficult  and  time  consuming,  but  the  payoff  in  information  provided  can  be 
extremely  valuable. 

As  noted  by  Holton  (1996),  the  major  limitation  of  the  Kirkpatrick  model  is  that  it  is 
primarily  a  taxonomy  that  does  not  fully  identify  all  constructs  underlying  the  variables  of 
interest,  thus  making  validation  impossible.  The  implied  causal  relationships  between  each  level 
in  the  model  have  not  been  fully  demonstrated.  Moreover,  seldom  is  data  collected  on  all  four 
levels,  particularly  the  last  two  levels.  This  may  be  due  in  part  to  the  difficulty  in  collecting 
transfer  and  overall  organizational  performance  data.  Holton  argues  for  a  more  comprehensive, 
psychometrically  sound  evaluation  model  that  measures  the  major  intervening  variables  that 
affect  the  learning-transfer  process.  Holton,  Bates,  and  Ruona  (2000)  use  the  term  transfer 
system  which  is  defined  as  all  factors  in  the  person,  training,  and  organization  that  influence 
transfer  of  learning  to  job  perfonnance.  Accurate  description  and  assessment  of  those  factors 
affecting  transfer  of  training  is  critical  because  it  helps  trainers  move  beyond  the  question  of 
whether  the  training  works  to  a  deeper  understanding  of  why  the  training  was  effective. 

Attempts  at  measuring  factors  affecting  transfer  have  been  problematical.  The  measures 
used  have  ranged  from  single-item  scales  to  multiple-item  content-validated  but  situation- 
specific  scales.  According  to  Holton,  Bates,  and  Ruona  (2000),  this  presents  several  problems. 
First,  the  tendency  toward  custom-designed  scales  for  each  effort  makes  generalization  of 
findings  across  studies  suspect  and  conclusions  about  the  underlying  structure  of  the  transfer 
climate  difficult.  Second,  empirical  evidence  demonstrating  the  validity  of  the  scale  constructs 
either  are  not  perfonned  or  are  not  included  as  part  of  the  effort.  Finally,  scales  employing 
single  item  measures  have  questionable  psychometric  properties. 
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The  development  of  a  set  of  transfer  system  scales  with  validated  constructs  and  known 
psychometric  qualities  would  enhance  the  chances  of  conducting  meaningful  cross-study 
comparisons,  add  significantly  to  understanding  the  transfer  process  (i.e.,  the  how  and  the  why) 
and  provide  significant  diagnostic  potential.  For  example,  the  scales  could  be  used  to  identify 
when  an  organization  is  ready  for  a  training  intervention  and  provide  information  to  guide 
pretraining  interventions  designed  at  increasing  training  effectiveness  (Holton,  Bates,  &  Ruona, 
2000). 


Training  system  inventory.  To  address  the  challenges  described  earlier,  Holton  and  his 
colleagues  have  developed  a  validated  instrument  for  learning  transfer  (Holton,  2008),  the 
Learning  Transfer  System  Inventory  (LTSI).  The  LTSI  contains  89  items  assessing  16  factors 
which  include  two  construct  domains:  ‘Training  in  Specific’  and  ‘Learning  in  General.’  The 
first  domain  contains  63  items  measuring  1 1  constructs  representing  factors  impacting  the 
training  program  attended.  The  remaining  items  (26)  and  constructs  (5)  assess  general  factors 
that  are  not  program  specific.  Tables  1  and  2  list  all  16  constructs  and  their  definitions. 

Table  1 

Learning  Training  System  Inventory:  Training  Specific  Scales 


Factor 

Definition 

Training  Specific  Scales 

Learner  Readiness 

Extent  to  which  individuals  are  prepared  to  enter  and 
participate  in  training. 

Motivation  to  Transfer 

Direction,  intensity,  and  persistence  of  effort  toward  utilizing  in 
a  work  setting  skills  and  knowledge  learned. 

Positive  Personal  Outcomes 

Degree  to  which  applying  training  on  the  job  leads  to  outcomes 
that  are  positive  for  the  individual. 

Negative  Personal  Outcomes 

Extent  to  which  individuals  believe  that  not  applying  skills  and 
knowledge  learned  in  training  will  lead  to  negative  personal 
outcomes. 

Personal  Capacity  for  Transfer 

Extent  to  which  individuals  have  the  time,  energy  and  mental 
space  in  their  lives  to  make  changes  required  to  transfer 
learning  to  the  job. 

Peer  Support 

Extent  to  which  peers  reinforce  and  support  use  of  learning  on 
the  job. 

Supervisor  Support 

Extent  to  which  supervisors/managers  support  and  reinforce 
use  of  training  on  the  job. 

Supervisor  Sanctions 

Extent  to  which  individuals  perceive  negative  responses  from 
supervisors/managers  when  applying  skills  learned  in  training. 

Perceived  Content  Validity 

Extent  to  which  trainees  judge  training  content  to  accurately 
reflect  job  requirements. 

Transfer  Design 

Degree  to  which  1)  training  has  been  designed  and  delivered 
to  give  trainees  the  ability  to  transfer  learning  to  the  job,  and  2) 
training  instructions  match  job  requirements. 

Opportunity  to  Use 

Extent  to  which  trainees  are  provided  with  or  obtain  resources 
and  tasks  on  the  job  enabling  them  to  use  training  on  the  job. 
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Table  2 

Learning  Training  System  Inventory:  General  Scales 


Factor 

Definition 

General  Scales 

Transfer  Effort  -  Performance 
Expectations 

Expectations  that  effort  devoted  to  transferring  learning  will 
lead  to  changes  in  job  performance. 

Performance  -  Outcomes 
Expectations 

Expectation  that  changes  in  job  performance  will  lead  to 
valued  outcomes. 

Resistance/Openness  to 
Change 

Extent  to  which  prevailing  group  norms  are  perceived  by 
individuals  to  resist  or  discourage  the  use  of  skills  and 
knowledge  acquired  in  training. 

Performance  Self-Efficacy 

An  individual’s  general  belief  that  they  are  able  to  change  their 
performance  when  they  want  to. 

Performance  Coaching 

Formal  and  informal  indicators  from  an  organization  about  an 
individual’s  job  performance. 

The  LTSI  enables  trainers  to  more  definitively  answer  questions  about  the  nature  of 
learning  transfer  in  the  workplace  and  about  barriers  and  enablers  of  transfer.  However,  it  does 
not  measure  transfer  directly,  which  limits  the  types  of  inferences  that  can  be  made  concerning 
relationships  with  transfer  outcomes  (Burke  &  Hutchins,  2007). 

Assessing  transfer.  To  fairly  assess  the  impact  of  training  interventions  and  related 
variables  on  transfer,  transfer  outcomes  must  be  directly  assessed.  Burke  and  Hutchins  (2007) 
reported  in  their  review  very  few  instances  where  researchers  have  directly  assessed  transfer 
outcomes  (trainee  behavior-performance).  The  majority  of  studies  have,  instead,  assessed  trainee 
self-reports  of  transfer  intentions  and  motivations,  e.g.,  motivation  to  learn,  motivation  to 
transfer.  High  ratings  on  these  variables  do  not  necessarily  mean  that  the  trainees  have,  in  fact, 
transferred  learning  back  to  the  job. 

While  direct  assessment  of  transfer  remains  the  gold  standard,  obtaining  these  measures 
as  well  as  measures  of  organizational  effectiveness  presents  significant  challenges.  Once 
trainees  complete  training  and  return  to  their  jobs  it  becomes  very  difficult  and  costly  to  collect 
the  relevant  perfonnance  measures. 

Facteau  et  al.  (1995)  argue  in  favor  of  using  self-report  measures  and  indicate  that 
trainees  can,  in  fact,  accurately  self-report  training  transfer  outcomes.  In  fact,  self-reports  that 
assess  tangible  aspects  of  transfer  versus  “happiness  questions”  may  provide  an  inexpensive  first 
look  at  transfer  effectiveness  (Holton,  1996).  The  utility  of  self-report  data  can  be  enhanced  by 
moving  away  from  single  source  reports  from  trainees.  Facteau  et  al.  advocate  a  comprehensive 
assessment  approach  consisting  of  additional  measures  of  trainees’  skill  transfer  from  other 
sources,  such  as  ratings  made  by  subordinates,  peers  and  supervisors,  measures  of  learning  and, 
if  possible,  measures  of  behavior  change  resulting  from  training  and  organizational  results 
measures  (e.g.,  productivity,  absenteeism,  turnover).  Limitations  in  self-reports  can  also  be 
minimized  by  using  highly  specific  items,  and  by  emphasizing  anonymous  and  confidential 
surveys  (Facteau  et  al.,  1995).  Multiple  assessments  of  perfonnance  at  regular  intervals  up  to  12 
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months  has  also  been  proposed  as  an  additional  strategy,  in  concert  with  those  already 
mentioned,  for  providing  a  more  accurate  assessment  of  transfer  interventions  (Burke  & 
Hutchins,  2007). 

An  alternative  approach  for  assessing  transfer.  Bransford  and  his  colleagues  (e.g., 
Bransford  &  Schwartz,  1999;  Bransford,  Brown,  &  Cocking,  2000;  Schwartz,  Sears,  &  Chang, 
2008)  have  proposed  an  alternative  strategy  for  assessing  transfer  of  learning.  Bransford  and 
Schwartz  (1999)  argue  that  current  methods  of  measuring  transfer  work  well  for  studying  full¬ 
blown  expertise,  but  they  represent  too  blunt  of  an  approach  for  studying  the  smaller  changes  in 
learning  that  lead  to  the  development  of  expertise. 

They  note  that  one  of  the  key  features  of  most  transfer  studies  is  the  use  of  a  final  transfer 
task  that  involves  what  they  call  sequestered  problem  solving  (SPS).  In  SPS,  there  are  no 
opportunities  for  trainees  to  demonstrate  their  abilities  to  learn  to  solve  new  problems,  by 
seeking  help  from  other  resources  such  as  texts  or  colleagues  or  by  trying  things  out,  receiving 
feedback,  and  getting  opportunities  to  revise.  Central  to  the  SPS  paradigm  is  the  notion  of 
transfer  as  the  ability  to  directly  apply  one’s  previous  learning  to  a  new  setting  or  problem. 

These  conventional  “one-shot”  tests  often  seriously  underestimate  the  amount  of  transfer  that 
trainees  display  from  one  domain  to  another. 

Bransford  and  his  colleagues  (Bransford  &  Schwartz,  1999;  Bransford,  Brown,  & 
Cocking,  2000;  Schwartz,  Sears,  &  Chang,  2008)  expand  this  view  by  focusing  on  other  more 
subtle  indicators  of  transfer  including  speed  in  learning  a  new  domain  and  the  quality  of 
questions  asked  by  trainees  during  tests  of  transfer.  The  key,  from  their  perspective,  is  to  look 
for  evidence  of  initial  learning  trajectories.  So,  rather  than  an  evaluation  of  whether  people  can 
generate  a  finished  product,  the  focus  shifts  to  whether  they  are  prepared  to  leam  to  solve  new 
problems. 

A  more  sensitive  way  to  assess  the  degree  to  which  trainees’  learning  has  prepared  them 
for  transfer  is  to  use  methods  of  dynamic  assessment  such  as  “graduated  prompting”  (e.g., 
Campione  &  Brown,  1987).  This  strategy  can  be  used  to  assess  the  amount  of  help  needed  for 
transfer  by  counting  the  number  and  types  of  prompts  that  are  needed  before  trainees  are  able  to 
demonstrate  transfer.  For  example,  some  trainees  may  demonstrate  transfer  following  a  general 
prompt  such  as  “Can  you  think  of  something  you  did  earlier  that  might  be  relevant?”  Other 
trainees  may  require  prompts  that  are  more  specific.  According  to  Bransford,  Brown,  &  Cocking 
(2000),  tests  of  transfer  that  use  a  graduated  prompting  scheme  can  provide  a  more  fine  grained 
analysis  of  learning  and  its  effects  on  transfer  than  simple  one  shot  measures  that  assess  whether 
the  trainee  could  produce  the  final  product.  Bransford  and  Schwartz  (1999)  suggest  that  using  a 
dynamic  assessment  approach  of  a  trainee’s  ability  to  learn  over  a  period  of  time  (e.g.,  a  month) 
may  be  a  better  predictor  of  long  term  success  on  the  job  than  a  one  shot  SPS  assessment 
immediately  following  training  on  the  job.  Some  examples  of  dynamic  assessments  (provided 
by  Bransford  and  Schwartz)  might  include  looking  at  the  trainees’  abilities  to  leam  new  sets  of 
materials.  Are  they  using  what  they  know  to  define  learning  goals?  Are  they  carefully 
evaluating  new  information  rather  than  simple  assimilating  it  to  existing  schemas?  Are  they  able 
to  work  collaboratively  with  others?  Are  they  reaching  sound  conclusions  based  on  existing 
evidence?  Are  they  able  to  reflect  on  their  learning  processes  and  strategies?  Schwartz  and 
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Martin  (2004)  present  a  design  strategy  that  would  enable  trainers  to  conduct  dynamic 
assessments  to  determine  students’  preparation  for  new  learning  and  subsequent  transfer  of 
learning  to  new  situations. 

Training  for  transfer.  As  part  of  the  objective/goal  setting  process,  trainers  must 
determine  if  the  focus  of  the  training  is  to  improve  near  or  far  transfer  (Yamnill  &  McLean, 
2001).  Near  transfer  is  the  application  of  learning  to  situations  similar  to  those  in  which  initial 
learning  has  taken  place  (on  the  job).  Far  transfer  is  the  application  of  learning  to  situations 
dissimilar  to  those  of  the  original  learning  events.  According  to  Yamnill  and  McLean,  whether 
one  achieves  near  or  far  transfer  appears  to  be  dependent  on  which  theory  (Identical  Elements 
Theory  versus  Principles  Theory)  of  transfer  guides  the  development  and  presentation  of  the 
training  program.  According  to  the  theory  of  identical  elements,  transfer  is  improved  by 
increasing  the  degree  of  correspondence  among  the  training  setting  stimuli,  responses,  and 
conditions  and  those  related  factors  operative  in  the  performance  setting  (Thorndike  & 
Woodworth,  1901).  The  principles  theory  suggests  that  that  training  should  focus  on  the  general 
principles  necessary  to  learn  a  task  so  that  the  learner  can  apply  them  to  solve  problems  in  the 
transfer  environment  (Goldstein,  1986). 

Holton  and  Baldwin  (2003)  introduced  the  Transfer  Distance  Model  that  depicts  the 
concept  of  transfer  distance  (i.e.,  near  versus  far  transfer)  as  the  gap  between  the  learning 
environment  and  application  of  knowledge  and  skills  in  the  job  environment.  The  model 
provides  a  useful  means  of  integrating  the  notions  of  near  and  far  transfer  under  a  single 
continuum  (see  Table  3).  The  transfer  distance  concept  provides  practitioners  and  trainers  a 
useful  means  of  thinking  about  learning  transfer  systems  by:  1)  locating  the  type  of  learning 
event  on  the  continuum;  and  2)  locating  the  type  of  transfer  that  is  targeted. 

Table  3 

The  Transfer  Distance  Conceptual  Model  (Holton  &  Baldwin,  2003) 


Phase  1 

Learning  Process:  From  Knowledge  to  Performance  Capability 

• 

Event  1 .  Acquiring  knowledge  (“know  that”) 

• 

Event  2.  Acquiring  knowledge  for  use  (“know  how”) 

• 

Event  3.  Building  performance  capability  through  practice 

Phase  2 

Work  Process:  From  Performance  Capability  to  Sustained  Performance 

• 

Event  4.  Application  for  job-specific  proficiency  (near  transfer) 

• 

Event  5.  Repeating  and  maintaining  application 

• 

Event  6.  Generalizing  for  far  transfer 

The  transfer  distance  continuum  progresses  through  two  phases  and  six  key  events  that  are 
briefly  described  below.  The  first  phase,  moving  from  knowledge  to  performance  capability,  is 
the  learning  process  and  the  traditional  domain  of  training.  Event  1  represents  the  starting  point 
for  most  training,  acquiring  cognitive  knowledge  or  “know  that”.  For  transfer  to  occur,  requires 
that  learning  be  expanded  to  event  2,  acquiring  knowledge  for  how  to  use  the  learning,  or  “know 
how.”  According  to  Holton  and  Baldwin  (2003)  events  1  and  2  represent  the  minimum  learning 
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required  to  make  transfer  possible.  Event  3,  building  performance  capability  through  practice  is 
hypothesized  to  enhance  transfer  by  providing  trainees  opportunities  to  practice  what  they  have 
learned. 

The  second  phase  moves  the  learner  from  performance  capability  to  sustained 
performance  and  represents  the  work  process  using  the  learning  acquired  in  training.  Event  4 
symbolizes  the  traditional  notion  of  what  is  meant  by  near  transfer,  i.e.,  the  proficient  application 
of  learned  material  to  the  trainee’s  immediate  job.  Event  5  involves  repeating  and  maintaining 
learned  performance.  Event  6  represents  the  final  stage  of  the  (far)  transfer  process,  the 
generalization/adaptation  of  learning  to  jobs  or  tasks  not  originally  anticipated  by  the  training, 
but  related  in  a  way  that  allows  the  learning  effects  to  multiply. 

Enhancing  near  transfer.  The  following  recommendations  are  provided  for  increasing 
the  likelihood  of  near  transfer  (from  Yamnill  &  McLean,  2001): 

•  The  more  the  training  content  and  program  reflect  the  workplace,  the  more 
successful  the  near  transfer  (Baldwin  &  Ford,  1988). 

•  The  greater  the  specificity  about  where  and  how  the  training  is  to  be 
applied  to  the  job,  the  more  successful  the  near  transfer  (Clark  & 

Voogel,  1985). 

•  The  more  overlearning  of  the  task  is  encouraged,  the  more  successful 
the  near  transfer  (Noe,  1986). 

•  The  more  procedural  nature  of  the  task  is  emphasized,  the  more 
successful  the  near  transfer  (Clark  &  Voogel,  1985). 

•  The  more  the  application  of  the  training  is  restricted  to  only  those  areas 
for  which  the  trainees  are  prepared,  the  more  successful  the  near 
transfer  (Clark  &  Voogel,  1985). 

Near  transfer  would  be  the  objective  of  short-term  skill  development  that  can  be  applied 
immediately  to  improve  performance  in  one’s  current  position  (Spitzer,  1984).  Near  transfer  is 
most  critical  when  pursuing  technical  training  because  technical  training  (e.g.,  operating  a 
machine  lathe,  small  engine  repair)  usually  teaches  specific  behaviors  and  procedures  applicable 
to  the  individual’s  current  job  (Laker,  1990). 

Enhancing  far  transfer.  The  theory  of  transfer  through  principles  emphasizes  the 
importance  of  creating  variety  and  explaining  the  “why”  that  underlies  what  an  individual  is 
being  taught.  The  following  recommendations  are  provided  for  increasing  the  likelihood  of  far 
transfer  (modified  from  Yamnill  &  McLean,  2001): 

•  The  better  the  trainees  understand  the  underlying  principles,  concepts, 
and  assumptions  of  the  skills  and  behaviors  they  are  learning,  the  more 
successful  the  far  transfer  (Goldstein,  1986).  Continuous,  formative  feedback 
provided  throughout  training  is  critical  for  enhancing  deep  understanding  of  learning 
materials  (Bransford,  Brown,  &  Cocking,  2000). 

•  To  enhance  trainees’  understanding  of  critical  features  of  new  information, 
have  them  attempt  to  solve  a  representative  problem  first  to  allow  trainees  to 
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contrast  their  thinking  with  others,  including  experts  in  an  area,  and  identify 
knowledge  gaps  (Bransford  &  Schwartz,  1999;  Schwartz  &  Martin,  2004). 

•  Including  direct  instruction  in  training  interventions  can  improve  far  transfer 
by  providing  explanations  and  efficient  solutions  devised  by  experts.  To 
achieve  this  benefit  without  undennining  transfer,  direct  instruction  should  be 
provided  after  trainees  have  first  been  allowed  to  grapple  with  representative 
problems  (e.g.,  invent  or  generate  solutions).  This  sequencing  improves  deep 
understanding  by  affording  the  trainee  sufficient  time  to  fully  process  the 
underlying  structure  of  the  problem  and  prepares  them  to  better  appreciate  the 
deeper  implications  of  the  expert  solutions  (Schwartz  &  Martin,  2004; 

Schwartz,  2008). 

•  Scaffolding  initial  learning  using  contrasting  cases  and  worked  examples 
enhances  deep  understanding  and  prepares  trainees  for  future  learning  (far 
transfer)  (Bransford  &  Schwartz,  1999;  Schwartz  &  Martin,  2004;  Schwartz, 

Sears,  &  Chang,  2008). 

•  The  more  trainees  practice  in  different  contexts  and  use  novelty  in  their 
practice  exercises,  the  more  successful  the  far  transfer  (Baldwin  & 

Ford,  1988;  Goldstein,  1986). 

•  The  more  encouragement  trainees  receive  during  training  to  discuss  and 
apply  the  training  in  situations  of  their  own  choosing,  the  more 
successful  the  far  transfer  (Noe,  1986). 

•  The  more  encouragement  trainees  receive  after  training  to  apply  the 
training  to  situations  other  than  those  for  which  they  were  trained,  the 
more  successful  the  far  transfer  (Goldstein,  1986). 

•  The  more  efficient  trainees  become  in  monitoring/managing  their  learning 
strategies/goals  and  resources,  the  greater  the  likelihood  of  sustained  positive  (far) 
transfer  (Bransford,  Brown,  &  Cocking,  2000;  Wexley  &  Baldwin,  1986). 

Principles  theory  is  critical  to  far  transfer  because  knowledge  can  be  abstracted  and 
connected  to  new  problems.  If  trainees  can  understand  the  principles  and  concepts  and  if  they 
have  a  chance  to  practice  exercises  and  apply  situations  in  training  programs  to  their  workplace, 
they  are  more  likely  to  apply  their  newly  acquired  skills  and  behaviors  when  they  are  faced  with 
new  challenges  and  unfamiliar  problems.  Far  transfer  principles  may  be  most  appropriate  for 
higher  level  cognitive  skills  such  as  leader  development  or  creative  problem  solving 
(adaptability)  since  the  focus  is  more  to  prepare  trainees  to  better  deal  with  unspecified  problem 
areas  in  the  future. 

Near  and  far  transfer  can  be  viewed  as  a  series  of  goals  or  objectives  of  training  and 
should  be  reflected  in  the  content  and  design  of  training.  It  is  critical  to  identify  in  advance  the 
situations  in  which  training  is  to  be  applied.  Yamnill  &  McLean  (2001)  underscore  a  point  made 
earlier.  No  matter  how  well  designed  the  training,  the  training  program  must  directly  address 
clearly  identified  individual  and  organizational  problems.  The  relevance  of  the  knowledge, 
skills,  and  attitudes  taught  in  training  to  the  trainee’s  performance  on  the  job  is  critical  in 
determining  transfer. 
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Training  for  efficiency  versus  training  for  innovation.  Typically,  near  transfer  involves 
two  types  of  knowledge,  replicative  (declarative  knowledge)  and  applicative  (procedural).  By 
focusing  on  repetitive  practice  of  tasks  that  will  be  performed  back  on  the  job  and  providing 
experiences  with  classes  and  components  of  problems  so  that  they  become  “routine”  and  easy  to 
solve  later,  individuals  can  develop  high  levels  of  efficiency  in  task  performance.  However,  an 
overemphasis  on  efficiency  can  result  in  functionally  fixed  behaviors  which  may  restrict  transfer 
to  highly  similar  situations  and  limit  innovation  and  creativity  (Schwartz,  Bransford,  &  Sears, 
2005). 


Designing  environments  to  foster  innovation  and  creativity  (far  transfer)  requires  that 
individuals  have  the  opportunity  to  test  out  ideas,  modify  long  established  assumptions  or  beliefs 
about  events  or  processes  when  appropriate,  and  for  providing  interactions  that  can  reveal  new 
information  and  orient  learners  to  notice  it.  Schwartz  et  al.  (2005)  believe  that  it  is  important 
(from  an  educational  standpoint)  to  balance  efficiency  and  innovation  within  the  same  protocol. 
Balanced  instruction  would  include  opportunities  to  leam  with  understanding  as  well  as 
becoming  more  efficient  at  performing  the  requisite  processes  underlying  successful  execution  of 
the  task.  The  key  to  this  type  of  balanced  instruction  is  to  ensure  that  individuals  are  provided 
opportunities  to  experiment  with  ideas  and,  in  the  process,  experience  the  need  to  change  them. 
These  kinds  of  experiences  often  require  opportunities  to  interact  actively  with  people  and 
learning  resources  (Schwartz  et  al.  2005). 

Initial  research  (e.g.,  Schwartz  &  Bransford,  1998)  has  shown  that  one  of  the  most 
effective  design  strategies  for  achieving  both  qualities  entails  combining  opportunities  for 
innovation  -  developing  interpretive  knowledge  (e.g.,  analyzing  data  sets  and  graphing  the  results 
to  identify  ‘interesting’  patterns  in  the  data)  and  opportunities  for  learning  efficient  solutions 
invented  by  experts.  This  (learning  efficient  solutions)  was  typically  provided  in  the  context  of  a 
class  lecture  following  attempts  by  students  to  invent  (graphically  depict  key  findings  from  the 
data  sets).  Transfer  was  assessed  by  asking  subjects  to  make  predictions  about  the  outcome  of  a 
novel  experiment. 

Intervention  in  the  work  place.  To  enhance  the  transfer  process,  there  must  be  some 
form  of  intervention  following  the  training.  Axtell,  Maitlis,  and  Yearta  (1996)  examined  transfer 
of  learning  one  month  and  one  year  following  training.  They  found  that  trainee  rated  transfer  of 
training  at  one  month  was  a  significant  predictor  of  trainee-rated  transfer  after  one  year. 
Moreover,  trainee  rated  levels  of  skill  did  not  change  appreciably  between  one  month  and  one 
year,  suggesting  that  the  period  immediately  after  the  course  may  be  critical  in  laying  the 
foundations  for  future  skill  use.  This  indicates  that  leaders/supervisors  may  not  only  need  to 
provide  the  opportunity  and  encouragement  for  skill  use  but  also  need  to  provide  trainees  with 
training-related  goals  to  be  attained  in  the  first  month  back  at  work  (See  also  Wexley  &  Baldwin 
[1986]  concerning  the  importance  of  goal  setting  in  sustaining  performance.) 

The  issue  of  goal  setting  and  subsequent  rewards  highlights  another  point.  Goals  must  be 
tied  to  those  of  the  organization.  The  most  significant  gains  in  transfer  will  come  when  learning 
is  more  tightly  integrated  into  the  process  and  reward  systems  relevant  to  the  organization 
(Holton  &  Baldwin,  2003). 
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Practical  implications.  The  research  presented  provided  some  useful  guidelines  for 
enhancing  the  transfer  process.  Development  of  effective  training  strategies  requires  careful 
consideration  of  the  major  intervening  variables  affecting  transfer  including  individual,  training 
design  and  organizational  influences.  This  will  entail  continued  development  and/or  refinement 
of  existing  instruments  to  assess  these  factors  both  pre  (as  a  diagnostic)  and  post  training.  This 
will  enhance  trainers’/researchers’  abilities  to  identify  organizational  and  work  environment 
factors  negatively  impacting  the  learning/transfer  process  and  will  allow  for  more  effective 
tailoring  of  the  intervention.  Additionally,  from  a  diagnostic  perspective,  these  instruments 
could  provide  valuable  feedback  to  unit  organizational  leaders  on  training  climate  factors  that 
may  need  to  be  addressed  prior  to  training. 

Further  work  is  also  indicated  with  regard  to  assessing  transfer.  This  may  be  particularly 
important  for  assessing  far  transfer  effects.  Current  strategies  for  assessing  far  transfer  should  be 
scrutinized  (e.g.,  sequestered  problem  solving)  versus  more  refined  approaches  (e.g.,  dynamic 
assessments  conducted  over  time)  to  identify  the  potential  utility  of  these  alternative  strategies. 

Finally,  greater  attention  must  be  directed  to  understanding  how  to  develop  training 
interventions  to  enhance  deep  understanding  and  far  transfer.  Passive  approaches  involving 
lecture/Power  Point  presentations  or  “fire  hosing”  vast  amounts  of  information  to  trainees  with 
little  or  no  opportunity  to  practice,  reflect,  and  modify  new  behaviors  are  clearly  ineffective  for 
enhancing  transfer  of  learning.  Alternative  approaches  focusing,  for  example,  on  situating 
instruction  within  a  problem  (e.g.,  Merrill,  2002)  or  on  active  involvement  by  the  trainees  in  the 
learning  process  by  collaboratively  solving  problems,  (e.g.,  problem-  or  inquiry  based  learning 
(Hmelo-Silver,  Duncan,  &  Chin  [2007];  Hemelo-Silver,  [2004])  and  general  strategies  for 
enhancing  deep  understanding,  (e.g.,  sequencing  of  instructional  events  and  tasks  (Schwartz  & 
Bransford,  1998,  Schwartz  &  Martin,  2004;  Schwartz,  Bransford,  &  Sears,  2005;  Schwartz, 
2008)  may  offer  a  more  viable  solution  for  improving  far  transfer  of  learning.  While  these 
approaches  may  entail  more  initial  work  on  the  behalf  of  trainers  to  construct  the  appropriate 
training  materials,  the  long  term  benefits  may  warrant  the  effort. 

The  Army  Institutional  Environment 

The  Army  institutional  training  environment  is  viewed  as  being  “contained”  within  the 
external  environment  discussed  above  and  comprises  elements  associated  with  training 
development  and  design  and  with  instructor  preparation  and  instructor  integration  with  the 
institutional  environment.  In  the  area  of  training  development  and  design,  the  first  section  below 
treats  the  status  of  training,  especially  with  respect  to  dL,  in  proponent  schools, 
media/technology  concerns,  and  the  status  of  learning  science  in  general.  The  second  section 
treats  the  Anny’s  training  available  for  institutional  instructors  and  how  those  instructors  fit  into 
the  institutional  training  environment. 
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Training  Development  and  Design 

In  1999,  President  Clinton  issued  Executive  Order  13111  (President  of  the  United  States, 
1999),  which  outlined  a  plan  to  .  make  effective  use  of  technology  to  improve  training 
opportunities  for  Federal  Government  employees.”  President  Clinton  described  a  vision  for 
achieving  this  goal  through  the  collaboration  of  the  military,  government,  private  industry,  and 
academia.  In  addition  to  describing  broad  programmatic  goals,  some  general  responsibilities 
were  assigned  to  different  government  and  military  agencies.  For  example,  the  Department  of 
Defense  (DoD)  was  given  the  responsibility  for  developing  standards  of  effective  distributed 
learning,  while  the  Department  of  Labor  was  tasked  with  establishing  training  technology 
websites. 

Nearly  ten  years  later,  it  has  become  apparent  that,  while  there  have  been  many  strides 
towards  realizing  the  vision  outlined  in  the  Order,  in  many  ways  the  collaborative  spirit  of  the 
order  appears  to  be  withering.  The  services  within  the  DoD  have  worked  independently,  with 
little  evidence  of  collaboration,  to  develop  effective  distributed  learning  (dL)  programs.  Within 
the  Army,  this  responsibility  has  fallen  largely  to  the  Training  and  Doctrine  Command 
(TRADOC).  As  the  organization  responsible  for  recruiting,  educating,  and  training  Soldiers, 
TRADOC  is  the  logical  choice  to  manage  the  Army’s  dL  program.  However,  TRADOC  as  an 
organization  has  not  participated  in  any  extensive  collaboration  with  non- Army  organizations. 
TRADOC’s  not  being  listed  on  the  Advanced  Distributed  Learning  (ADL)  website’s  list  of  DoD 
component  offices  (http://www.adlnet.gov/Technologies/adlr/ AD LRDocuments/Comr>onent%20 
Proponent%20  Offices.aspx)  is  an  indication  that  there  are  collaborative  opportunities  that  have 
not  yet  been  actively  pursued. 

This  approach  may  be  particularly  troubling  given  the  major  challenge  presented  to 
TRADOC:  preparing  Soldiers  for  deployment  on  limited  resources.  Funding  challenges  in 
particular  seem  to  have  largely  driven  TRADOC’s  distributed  learning  policy.  For  example, 
TRADOC  initially  stipulated  that  for  any  hour  of  distributed  learning  developed,  a  corresponding 
hour  of  classroom  instruction  be  eliminated  (Marlow,  personal  communication,  2008).  While 
this  zero-sum  approach  is  an  effective  cost-cutting  measure  (one  of  the  advantages  of  distributed 
learning  over  classroom  learning  should  be  cost),  it  will  only  be  effective  in  achieving  the  goal  of 
providing  the  highest  quality  training  to  Soldiers  if  serious  thought  and  consideration  is  given  to 
the  science  of  learning.  Reducing  classroom  hours  is  an  administrative  challenge;  doing  so  while 
maintaining,  or  even  improving,  training  effectiveness  is  a  formidable  scientific  challenge. 

Even  as  a  cost-cutting  measure,  TRADOC’s  approach  may  be  non-optimal.  While 
distributed  learning  should  be  a  less  expensive  alternative  to  standard  training  through  the 
reduction  of  manpower  and  logistical  costs,  it  is  possible  that  the  Army  could  achieve  even 
greater  monetary  savings  by  collaborating  with  non- Army  organizations.  Such  organizations 
may  have  tools,  infrastructure,  and/or  processes  that  could  be  reused  or  shared  so  that  the  Army 
does  not  spend  money  “reinventing  the  wheel.”  There  is  one  notable  exploitation  of  this 
potential:  the  Ordnance  Corps’  Ordnance  Electronic  Maintenance  Training  Department  at  Ft 
Gordon,  GA,  and  the  Navy  Center  for  Combat  Systems  Unit  at  Naval  Station  Great  Lakes,  IL, 
utilize  a  common,  off-the-shelf  blended  learning  package  for  training  basic  electronics  (G. 
Holshouser,  personal  communication,  2008).  TRADOC  has  issued  Army-wide  dL  directives 


20 


that  do  not  mention  collaboration  with  non-Anny  organizations.  For  example,  the  purpose 
statement  in  ALARACT  126/2007  (Department  of  the  Army,  2007a)  describes  the  need  for  an 
Army- wide  process  for  developing,  managing,  registering,  and  delivering  courseware.  However, 
the  process  does  not  extend  to  include  consideration  of  other  services,  industries  or  institutions. 
Executive  Order  13111  initially  tasked  the  Department  of  Labor  with  leading  the  effort  to 
establish  training  technology  websites.  It  is  quite  possible  that  failing  to  exploit  non-Anny  dL 
efforts  may  result  in  unrealized  return  on  investment. 

Status  in  proponent  schools.  The  budgetary  constraints  that  TRADOC  faces  are  real 
and  formidable  and  emerge  from  the  actual  environmental  conditions  within  which  the  Army  is 
trying  to  develop  and  implement  a  distributed  learning  program.  The  purpose  of  this  section  of 
the  report  is  to  identify  the  status  of  current  Army  institutional  training,  and  to  provide  comment 
on  improving  training  effectiveness  using  distributed  learning.  The  approach  we  promote 
contrasts  with  the  zero  sum  approach  described  above,  in  which  distributed  learning  and 
classroom  learning  are  conceptualized  as  two  distinguishable  and  separate  parts  of  the  same 
whole.  Instead,  we  will  describe  a  “blended  learning”  approach  in  which  the  relationship 
between  distributed  learning  and  classroom-based  learning  is  emphasized  rather  than  the 
distinction  between  them;  the  two  reinforce  and  complement  each  other  rather  than  replace  each 
other.  We  believe  this  approach  can  help  to  save  money  while  improving  training  effectiveness. 

In  the  remainder  of  this  section,  we  present  our  findings  across  seven  training 
installations  from  interviews  designed  to  assess  the  current  state  of  Army  institutional  training. 
Based  on  these  findings,  we  provide  recommendations  on  how  to  improve  institutional  training 
by  implementing  a  blended  learning  approach.  These  recommendations  are  based  both  on  our 
findings  as  well  as  the  perspective  and  expertise  of  the  extended  research  team  with  their 
knowledge  and  experience  in  blended  learning  pedagogy,  training  development,  and  institutional 
training. 

Our  interview  results  describe  the  current  state  of  Army  distributed  learning.  Our  main 
finding  is  that  the  program  is  lacking  many  of  the  basic  capabilities  and  services  required,  and 
therefore  hampers  those  involved  in  dL  from  exploring  or  implementing  state-of-the-art 
approaches  to  training  development,  delivery,  and  management.  We  explore  this  finding  through 
a  number  of  common  themes  that  emerged  from  our  schoolhouse  interviews,  which  are  presented 
briefly  here  and  then  discussed  in  detail  in  our  conclusions. 

Schoolhouse  interviews.  Researchers  travelled  to  seven  Army  training  institutions  to 
conduct  interviews  with  a  variety  of  personnel  involved  in  distributed  learning.  The  institutions 
were: 


•  Army  Field  Artillery  School,  Ft  Sill,  OK 

•  Army  Medical  Training  Center,  Ft  Sam  Houston,  TX 

•  Army  Intelligence  School,  Ft  Huachuca,  AZ 

•  Army  Signal  Center,  Fort  Gordon 

•  Army  Maneuver  Support  Center,  Fort  Leonard  Wood 

•  Army  Aviation  Center  of  Excellence,  Fort  Rucker 

•  Army  Combined  Anns  Support  Command,  Fort  Lee 
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Each  visiting  research  team  consisted  of  two  or  three  interviewers  with  expertise  in 
training  and  education  and  military  service.  This  combination  of  backgrounds  provided  the 
pedagogical  and  operational  expertise  required  to  address  the  many  aspects  of  the  training 
problem.  This  approach  of  combining  the  technical  expertise  of  training  and  education  with  the 
practical  experience  of  those  who  have  served  in  the  military  created  an  atmosphere  which 
allowed  for  a  free  exchange  of  information  among  the  interviewers  and  interviewees. 

Interview  materials.  The  research  team  developed  a  questionnaire  to  guide  interview 
discussion  (see  Appendix  A).  The  intended  use  of  the  questionnaire  was  not  to  provide  a  strict 
procedure  of  when  to  ask  which  questions,  as  certain  questions  would  not  be  applicable  to  all 
interviewees.  Rather,  the  questionnaire  was  intended  to  be  used  as  a  guide  for  interviewers  to 
ensure  that  they  were  addressing  key  topics  within  the  course  of  the  interview  discussion.  These 
topics  focus  on  course  development,  course  evaluation,  and  general  lessons  learned. 

Participants.  In  general,  participants  represented  a  wide  range  of  backgrounds  including 
training  development,  management,  administration,  and  contracting.  Participants  were  self- 
selected  based  on  their  availability  when  the  research  team  was  on  post.  The  team  interviewed 
anywhere  from  one  to  eight  participants  at  any  given  schoolhouse. 

Procedure.  Interviews  were  conducted  at  convenient  locations  for  the  interviewees  at  the 
training  institutions.  In  some  cases,  interviews  were  conducted  in  classrooms  or  conference 
rooms.  In  others,  interviews  were  conducted  in  offices  or  common  seating  areas.  The  length  of 
the  interviews  depended  entirely  on  the  availability  of  the  interviewees.  Several  interviews 
lasted  approximately  30  minutes,  while  others  lasted  two  hours,  or  even  over  the  course  of  a  day 
(in  cases  of  group  discussion).  Interviewers  recorded  notes  in  real-time  via  laptop  computers  or 
handwritten  notes. 

Interview  results.  Analysis  of  the  interviews  resulted  in  a  number  of  common  challenges 
facing  Army  dL  programs.  These  challenges  affected  all  of  those  involved  in  dL,  from 
administrators  to  instructors  to  training  developers  to  contract  administrators.  These  themes  are 
presented  below  and  then  discussed  in  greater  detail  independently. 

•  The  Army  is  facing  basic  technological  issues  that  impact  the  dL  program. 

•  The  Army  is  facing  basic  pedagogical  issues  that  impact  the  dL  program. 

•  The  Army  is  facing  basic  organizational  issues  that  impact  the  dL  program. 

The  Army  is  facing  basic  technological  issues  that  impact  the  dL  program.  Perhaps  most 
crucial  to  a  successful  dL  program  is  the  necessary  infrastructure  to  support  the  program.  Two 
of  the  key  functionalities  that  the  infrastructure  supports  are  1)  the  student’s  ability  to  access 
material,  and  2)  the  instructor’s  ability  to  update  material.  This  is  a  most  basic  and  fundamental 
need,  yet  it  has  not  been  satisfied  in  current  Army  training  institutions.  The  consequences  of  a 
inadequate  infrastructure  are  both  immediate  and  far-reaching.  Just  as  it  is  theorized  that  human 
beings  must  meet  basic  needs  (food,  security)  before  achieving  higher-level  ones  (e.g.,  self 
actualization)  (e.g.,  Maslow,  1943),  a  dL  program’s  ability  to  achieve  higher-level  goals  depends 
on  its  ability  to  first  achieve  basic  ones.  Because  the  basic  infrastructure  is  not  in  place  to 
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support  basic  dL  infrastructure  needs,  Army  training  institutions  have  not  had  the  opportunity  or 
the  resources  to  address  higher-level  training  issues  such  as  training  evaluation,  theoretical 
training  approaches,  appropriate  media  selection,  and  so  on. 

In  any  dL  environment,  one  key  component  of  a  solid  infrastructure  is  system  response 
time,  or  the  end-user’s  perceived  time  lag  between  input  and  completed  response.  In  general 
terms,  system  response  time  for  on-line  dL  has  three  components:  client  (local  PC)  processing 
time,  transport  (network)  lag,  and  server  processing  time.  As  system  response  time  worsens, 
end-users  experience  longer  and  longer  times  for  their  inputs  to  be  acknowledged  and  for  pages 
to  update  or  download. 

Interviewees  repeatedly  cited  system  response  time  as  being  a  major  restriction  on  their 
ability  to  not  only  deliver  dL,  but  also  to  develop  it.  The  immediate  effect  of  increased  system 
response  time  on  delivery  (e.g.,  slowed  or  failed  delivery)  is  somewhat  obvious.  Many  trainees 
find  that  they  have  better  response  times  outside  classroom  hours  at  their  homes  than  when  they 
are  in  schoolhouses.  One  reason  for  this  may  be  that  the  current  infrastructure  is  such  that  all 
servers  for  dL  are  based  out  of  a  single  location,  Fort  Eustis.  As  such,  Fort  Eustis’s  servers  are 
being  accessed  by  all  military  personnel  seeking  dL.  This  strain  on  the  system  often  results  in 
very  slow  loading  times.  Interviewees  described  that  during  peak  hours  (usually  after  1000), 
loading  times  through  the  ALMS  can  be  extremely  slow  (they  recall  instances  of  ten  minute 
loading  times  for  a  single  page).  Furthermore,  if  Fort  Eustis  experiences  any  problems  at  their 
facilities,  the  entire  Anny  suffers.  Interviewees  recalled  a  time  when  Fort  Eustis  closed  during 
severe  weather,  and  no  Army  installations  were  able  to  conduct  regular  dL  training. 

However,  the  effect  of  long  system  response  time  on  training  development  has  an 
unexpected  and  far-reaching  consequence.  Interviewees  described  that  they  intentionally 
develop  training  in  a  different  way  than  they  would  actually  want  to,  just  to  develop  a  product 
that  works.  In  other  words,  developers  are  being  forced  to  make  tradeoffs  between  pedagogy 
and  functionality.  For  example,  training  media  selection  is  sometimes  compromised  when  more 
sophisticated  media  (videos,  simulations),  though  desired  and  pedagogically  appropriate  in 
certain  situations,  are  not  selected  because  developers  know  that  on-line  training  using  real-time 
interactions  and  extensive  downloaded  content  will  become  frustrating  to  the  learner  under 
conditions  of  long  system  response  time.  As  a  result,  some  interviewees  felt  obligated  to 
develop  simple  “page-turner”  courseware  because  it  has  the  best  chance  of  running  properly. 

In  addition  to  system  response  time  considerations,  there  are  other  factors  that  affect  a 
student’s  ability  to  access  training.  For  example,  the  student  may  not  have  a  computer  at  work,  or 
may  not  have  permissions  to  install  the  software  or  some  component  to  run  the  distributed 
learning  material.  If  the  material  is  web-based,  the  student  may  not  have  permissions  to  the  site 
where  it  is  located  due  to  AKO  security,  or  may  not  be  able  to  find  it  easily  or  repeatedly.  The 
material  may  have  been  created  using  a  program  that  is  no  longer  compliant  with  current  web 
browsers.  Additionally,  the  local  Department  of  Information  Management  (DOIM)  may  place 
additional  restrictions  on  websites  or  firewalls  that  prevent  the  student  from  accessing 
supplementary  material  on  the  internet.  This  last  example  was  repeatedly  discussed  during 
interviews.  For  example,  at  one  time,  students  at  Fort  Sam  Houston  were  not  able  to  search  for 
sites  using  the  terms  “anatomy  and  physiology”  even  though  they  are  studying  medicine. 
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A  second  critical  component  of  dL  infrastructure  is  a  Learning  Management  System 
(LMS).  A  LMS  is  responsible  for  managing  and  delivering  training  as  well  as  tracking  student 
progress  and  results.  The  Anny  is  developing  the  Army  Learning  Management  System  (ALMS) 
to  function  as  its  own  LMS.  The  ALMS  has  undergone  several  changes,  and  continues  to  do  so. 
Interviewees  expressed  major  dissatisfaction  with  previous  and  current  versions  of  the  ALMS. 

Interviewees  described  how,  in  the  past,  both  contractors  and  in-house  developers  often 
submitted  courseware  that  failed  ALMS  compliance  testing  by  the  Army  Training  Support 
Center  (ATSC)  even  though  they  had  developed  courseware  to  specifications.  There  were 
instances  when  courseware  passed  SCORM 1  2004  compliance  testing,  but  failed  ALMS  testing. 
Interviewees  were  encouraged  that  the  Army  recently  agreed  to  add  Blackboard  to  its  list  of 
approved  systems  to  work  with  the  ALMS.  As  an  established  LMS,  dL  developers  have  found 
that  Blackboard  is  much  more  successful  delivering  courseware  (already  an  improvement  over 
the  ALMS)  and  even  promotes  some  collaboration  functionalities,  both  among  trainees  and 
between  trainees  and  trainers.  To  the  extent  possible,  developers  want  to  capitalize  on 
Blackboard  because  of  its  reliability. 

Instructors  also  suffer  from  basic  infrastructure  shortcomings.  Instructors  update  their 
courses  every  time  they  teach  the  infonnation.  They  are  constantly  learning  from  each  other 
about  which  techniques  worked  well  to  explain  a  concept  and  which  did  not.  They  are  also 
adapting  to  new  doctrine  as  it  is  published,  such  as  FM  3-0,  Operations  (Department  of  the 
Army,  2008).  FM  3-0  has  had  a  significant  impact  on  the  students  in  the  Field  Artillery 
Captain’s  Career  Course,  yet  the  instructors  who  teach  the  distributed  learning  portion  to  the 
reserve  component  do  not  have  the  ability  to  easily  update  or  change  the  material.  Therefore, 
they  must  commit  part  of  the  reserve  component  two  week  resident  portion  of  the  course  to  help 
the  students  “unlearn”  the  old  concepts  and  grasp  the  new  ones. 

In  sum,  the  lack  of  adequate  infrastructure  has  both  immediate  and  far-reaching 
consequences  for  the  dL  program.  Long  loading  times  or  failed  delivery  negatively  impact  the 
training  experience.  Furthermore,  dL  developers  restrict  their  pedagogical  approaches  based  on 
expected  delivery  failures.  Lastly,  for  instances  in  which  training  is  developed  and  delivered, 
there  is  no  simple  and  efficient  way  to  update  existing  training  material. 

The  Army  is  facing  basic  pedagogical  issues  that  impact  the  dL  program.  An  effective  dL 
program  requires  not  just  the  necessary  technology,  but  also  the  necessary  pedagogy  required  for 
effective  training.  It  appears  that  much  of  the  guidance  given  to  instructors  and  training 
developers  has  been  in  the  form  of  policies  rather  than  education.  The  consequence  in  some 
cases  is  that  reaction  to  the  policy  inhibits  education  on  pedagogy. 

As  described  above,  TRADOC  has  gravitated  to  dL  in  part  because  of  the  potential  for 
cost  savings  during  a  period  of  financial  strain.  Some  costs  are  reduced  by  eliminating  the 
logistical  costs  of  transporting  units  to  training  installations,  including  providing  transportation, 
lodging,  and  food.  However,  some  costs  are  also  reduced  by  reducing  instructor  hours.  Recall 
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that  TRADOC  mandated  that  a  dL  hour  gained  should  correspond  to  an  instructor  hour 
eliminated.  Consequently,  many  instructors  feel  threatened  by  dL  in  a  number  of  ways. 

First  and  foremost  is  the  threat  of  job  loss.  While  eliminating  jobs  is  not  always  a  goal 
when  implementing  a  dL  program,  instructors  are  aware  it  can  be  an  unintended  side  effect.  The 
perceived  threat  of  job  loss  is  clearly  not  going  to  endear  dL  to  some  instructors.  Second, 
instructors  may  feel  put  upon  by  having  others  tell  them  how  to  run  their  course.  Instructors 
expressed  that  they  have  invested  significant  time  and  thought  into  their  curricula,  and  the 
demand  to  do  things  differently  may  not  be  well-received. 

Compounding  the  perceived  threats  dL  presents  to  developers,  designers,  and  instructors 
is  the  lack  of  guidance  and  education  they  have  received  on  dL  in  general  and  the  lack  of 
guidance  on  how  to  incorporate  it  into  their  classes.  Without  any  guidance  and  education,  few 
see  how  they  might  use  dL  to  their  advantage  in  a  course,  for  example  as  a  remediation  tool  or  as 
preparation  for  a  subsequent  classroom  exercise,  especially  in  cases  where  classroom  time  is 
reduced.  Instead,  in  order  to  satisfy  the  dL  mandate,  they  often  assign  lower  priority  topics  to  be 
developed  into  dL.  These  topics  tend  to  address  human  resources  issues  (sexual  harassment,  for 
example)  rather  than  focal  points.  This  reinforces  the  “zero  sum  approach”  in  that  it  emphasizes 
the  distinction  between  what  is  taught  via  dL  and  what  is  taught  in  the  classroom  rather  than  the 
potential  relationship  between  the  two.  But  furthermore,  it  may  negatively  impact  student 
perceptions  of  dL  if  they  are  aware  that  the  types  of  topics  that  are  taught  via  dL  may  not  be  very 
highly  valued. 

If  instructors  are  not  receiving  education  on  incorporating  dL  into  their  classrooms,  they 
could  consult  with  in-house  dL  specialists.  However,  instructors  are  sometimes  unaware  of  the 
support  that  is  available  to  them.  If  they  do  become  aware  of  the  in-house  services  and  seek  out 
personnel,  they  are  often  unprepared  for  delivering  the  materials  that  in-house  personnel  require 
to  develop  dL.  Instructors  seem  to  think  that  the  same  personal  computer  (PC)  presentation 
slides  that  they  use  for  lectures  can  be  used  for  dL.  In-house  personnel  must  then  explain  how 
instructors  will  not  be  present  to  “talk  to  the  slides”,  and  must  therefore  adhere  to  certain  formats 
and  standards  (e.g.,  writing  out  learning  objectives,  key  teaching  points,  etc.)  that  promote 
asynchronous  learning. 

In  many  instances,  creating  the  distributed  learning  product  has  not  followed  the 
traditional  TR  350-70  (U.S.  Army  Training  and  Doctrine  Command,  1999)  approaches.  The 
training  developers  have  not  built  the  storyboards  or  determined  how  the  dL  product  will  fit  into 
the  overall  training  plan  for  the  target  audience.  Often  a  desire  to  have  the  latest  technology,  or 
Level  4  interactive  multimedia  instruction,  increases  cost  with  little  demonstrated  benefit. 
Instructors  at  Ft.  Sill  indicated  that  one  highly  interactive  product  allowed  the  student  to  practice 
interacting  with  an  aiming  circle,  and  this  was  very  helpful  as  part  of  the  overall  instruction. 

They  contrasted  that  with  another  program  that  was  to  help  commanders  with  training 
management,  and  the  program  replicated  a  commander’s  inbox.  While  aiming  is  critical  in  the 
first  example,  dealing  with  your  inbox  is  only  a  small  and  minimally  relevant  aspect  of  training 
management.  Some  tasks  are  better  reinforced  through  practice,  but  the  subject  matter  or  method 
of  interaction  can  make  those  tasks  tedious  or  less  effective,  thereby  reducing  the  overall  value  of 
the  dL  product. 
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Instructors  do  not  have  a  good  understanding  of  the  requirements  to  develop  distributed 
learning.  At  Ft.  Sam  Houston,  they  will  often  provide  a  set  of  slides  without  any  additional 
material  to  the  training  development  group.  They  are  not  aware  of  the  need  for  the  training 
development  to  be  done  in  painstaking  detail,  because  when  the  product  is  being  delivered,  an 
instructor  will  not  be  able  to  clarify  questions  the  student  might  have.  Many  instructors  have  not 
received  much  training  in  TR  350-70’s  traditional  training  development  processes,  and  while 
they  have  the  requisite  subject  matter  expertise,  they  sometimes  balk  at  the  suggestions  of  the 
training  development  community. 

In  sum,  Instructors  always  aim  to  update  their  courses  to  provide  the  most  effective 
training  possible.  In  addition  to  facing  technological  challenges  to  updating  their  materials,  they 
face  pedagogical  challenges  as  well  due  to  the  lack  of  training  they  have  received  on 
incorporating  dL  into  their  classrooms. 

The  Army  is  facing  basic  organizational  issues  that  impact  the  dL  program.  There  are 
organizational  challenges  that  are  preventing  an  integrated  approach  to  distributed  learning. 
Because  training  developers,  information  technology  specialists,  and  subject  matter  experts  all 
typically  belong  to  different  organizations  and  have  demands  independent  of  their  consideration 
of  distributed  learning,  it  is  difficult  to  form  and  maintain  a  team  of  individuals  who  represent 
the  various  stakeholders  who  could  influence  a  change  in  the  approach  to  distributed  learning. 

Distributed  learning  requires  a  multifunctional  team  that  includes  TRADOC  leadership, 
government  contracting  administrators,  instructors  and  subject  matter  experts,  training 
developers,  training  delivery  specialists,  graphic  artists,  computer  programmers,  and  a  multitude 
of  infrastructural-focused  information  technology  specialists  at  various  installations.  Organizing 
a  team  with  numerous  stakeholders  is  a  difficult  undertaking.  However,  internal  processes,  with 
their  own  inherent  frictions,  can  further  obscure  the  possible  solution  space. 

Leaders  have  not  seen  an  example  of  how  distributed  learning  has  quickly  provided  the 
Soldier  the  knowledge,  skills,  or  abilities  he  needs  to  measurably  improve  performance.  For  this 
reason,  many  are  skeptical  of  the  value  of  distributed  learning.  GEN  Wallace,  at  the  Army 
Training  Support  Center  (ATSC)  conference  on  distributed  learning  in  March  of  2008  stated,  “I 
am  neither  a  proponent  nor  an  opponent  of  distributed  learning.”  Some  leaders  view  distributed 
learning  only  as  a  method  of  reducing  classroom  time  rather  than  an  additional  means  to  convey 
concepts  to  students  to  maximize  the  value  of  the  classroom  instruction  time.  This  limited  view 
of  distributed  learning  only  exacerbates  the  decision  making  process.  With  regard  to  distributed 
learning,  decision  makers  are  more  likely  to  consider  the  impact  on  the  workload,  the  desires  of 
their  bosses,  or  their  own  experience  with  distributed  learning  products  over  the  most  practical  or 
effective  approach  to  learning. 

TRADOC  branch  proponents  are  suffering  from  a  lack  of  manpower  to  manage, 
participate  in,  and  validate  distributed  learning  projects  that  have  been  contracted  to  civilian 
companies.  In  the  same  way  the  impoverished  technological  infrastructure  can  have  far-reaching 
consequences  for  dL  (e.g.,  in  affecting  training  media  selection,  as  described  above),  so  too  can 
personnel  components  of  infrastructure.  Manpower  shortages  often  place  major  burdens  on 
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current  government  contract  administrators.  They  must  monitor  more  contracts  than  is  optimal, 
and  consequently  cannot  devote  the  appropriate  attention  to  any  single  project.  Furthermore,  a 
shortage  of  subject  matter  experts  (SMEs)  places  the  government  at  a  disadvantage.  SME’s  are 
required  throughout  the  process  to  ensure  that  the  contractor  has  the  necessary  govermnent 
furnished  information  and  is  providing  the  desired  product.  They  must  review  and  contribute  to 
dL  development  in  order  for  a  product  to  be  integrated  into  a  broader  program  of  instruction. 
During  a  dL  contract,  the  government  and  the  contractor  will  typically  agree  to  a  detailed 
delivery  schedule  that  includes  cycles  of  Alpha  and  Beta  reviews  of  courseware.  If  the 
government  cannot  (a)  review  the  work  in  the  agreed  upon  timeframe,  or  (b)  review  the  work  in 
great  enough  detail  to  provide  quality  feedback,  then  the  contractor  has  grounds  to  proceed  with 
development  and  delivery  even  if  the  government  is  not  satisfied  with  the  quality  of  the  product. 
In  combination  with  an  ill-conceived  dL  subject,  the  contractor  may  deliver  a  product  that  is 
useless.  One  contracting  officer  at  Ft.  Sill  indicated  that  while  a  company  was  developing  a  dL 
product  for  the  Advanced  Field  Artillery  Targeting  and  Direction  System  (AFATADS),  the 
AFATADS  program  underwent  two  software  upgrades,  one  of  which  had  significant  impact  on 
the  user  interface.  This  resulted  in  a  product  that  was  instantaneously  obsolete.  Straus,  Shanley, 
Burns,  Waite,  and  Crowley  (2009)  present  a  more  detailed  analysis  of  issues  surrounding  dL 
contracting  and  assessment  of  dL  products. 

Training  media  selection.  Training  media  that  work  well  for  traditional  classroom 
learning  may  not  always  translate  well  into  a  distance  learning  environment.  A  basic  approach  to 
this  selection  criterion  is  to  involve  the  school  and  appropriate  personnel  to  determine  what 
material  cannot  be  taught  via  dL,  rather  than  to  decide  what  can  be  taught  using  dL.  Application 
of  this  method  will  result  in  a  blended  learning  environment  which  will  accommodate  for 
material  that  is  most  effective  when  delivered  through  residential  classroom  instruction  and 
material  that  is  compatible  with  the  distance  learning  mode  of  delivery. 

There  are  several  potential  advantages  and  disadvantages  of  traditional  classroom 
learning  and  distance  learning.  Advantages  of  traditional  classroom  learning  can  include 
immediate  feedback,  familiarity  of  this  approach  to  students  and  instructors,  motivation  to 
students,  and  cultivation  of  a  social  community  (Zhang,  Zhao,  Zhou,  &  Nunamaker,  2004).  On 
the  other  hand,  disadvantages  of  traditional  classroom  learning  can  include  being  instructor- 
centered,  time  and  location  constraints,  and  higher  costs.  The  potential  advantages  of  distance 
learning  include  being  learner-centered  and  self-paced,  being  flexible  with  regards  to  time  and 
location,  potentially  being  available  to  global  audience,  providing  unlimited  access  to  knowledge 
resources,  and  archival  capability  for  knowledge  reuse  and  sharing.  Alternatively,  disadvantages 
of  distance  learning  can  include  the  lack  of  immediate  feedback  in  asynchronous  e-learning, 
increased  preparation  time  for  the  instructor,  and  the  fact  that  dL  is  not  familiar  and  comfortable 
to  some  people,  which  could  bring  about  frustration,  anxiety,  and  confusion. 

In  order  to  select  the  appropriate  delivery  medium,  course  designers  must  identify  the 
required  instruction  methods  for  the  learning  objectives  and  the  medium  that  can  be  the  most 
cost  effective  for  delivery  (Abell,  2007).  When  doing  so,  course  designers  should  consider  the 
appropriate  training  delivery  at  the  learning  module  level  rather  than  at  the  course  level 
(Quinkert,  Morrison,  Fletcher,  Moses,  Roberts,  2007).  Three  criteria  for  considering  whether 


27 


instructional  material  could  be  delivered  through  distance  learning  or  on-site/residential  learning 
include: 

1)  If  sensory  input,  beyond  audio  and  visual,  is  necessary  (i.e.,  taste,  touch,  smell),  a  face- 
to-face  environment  is  suggested 

2)  If  complex  conditions  are  required  for  instruction  and  electronic  multimedia  cannot 
depict  the  complexity  of  the  conditions,  a  face-to-face  environment  is  suggested 

3)  If  an  instructor  must  observe,  evaluate,  and  provide  feedback  to  a  learner  who  is 
engaged  in  whole-task  practice  of  a  complex  task,  distance  learning  can  be  used  if  it 
allows  synchronous  observation  of  the  learner  and  audio  and  visual  feedback. 

In  the  case  when  either  environment  could  be  effective,  the  course  designer  should 
account  for  motivation,  media  compatibility,  and  technology  feasibility  and  utilize  the  four-step 
process  below  to  capitalize  on  the  best  media  for  the  type  of  learning  environment  (Yelon,  2006): 

1)  Think  about  purpose  and  strategies  of  training  methods. 

2)  Assess  motivation  to  restructure  instruction. 

3)  Check  media  compatibility. 

4)  Account  for  resources. 

After  selecting  the  method  to  deliver  the  training,  whether  via  school  house  or  dL,  the 
training  media  to  deliver  the  instruction  should  be  selected.  There  is  very  little  evidence  that 
supports  the  notion  that  some  media  are  better  than  others  for  delivering  training  (Clark,  2001). 

In  contrast,  research  suggests  it  is  the  quality  of  the  instructional  methods  employed  that  most 
influences  learning.  Therefore,  it  is  important  to  match  the  appropriate  training  media  with  a 
compatible  instructional  strategy. 

There  are  several  types  of  training  media  that  could  be  utilized  for  both  in-classroom 
instruction  and  DL.  For  in-classroom  instruction,  trainers  may  consider  four  traditional  modes  of 
media:  print,  audio,  television,  and  computers  (Holden  &  Westfall,  2005).  With  each  medium 
there  are  strengths,  weaknesses,  and  types  of  instruction  in  which  specific  media  are  most 
effective.  Table  4  summarizes  the  strengths,  weaknesses,  and  the  types  of  appropriate 
instructional  strategies  and  deliveries  for  the  four  traditional  modes  of  media  (Holden  & 

Westfall,  2005).  This  table  can  be  used  to  determine  the  conditions  under  which  to  use  the 
designated  media  type. 
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Table  4 

Properties  of  Traditional  Media  (adapted  from  Holden  &  Westfall,  2005). 
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There  are  several  modes  of  training  media  for  dL  instruction:  classroom  training, 
web-based  training,  computer-based  training,  webinars,  websites,  video,  simulations,  and 
chat  and  discussion  communities.  In  addition  to  these  different  training  media,  traditional 
media  used  for  classroom  instruction  may  be  used  for  dL  (Simonson,  2007).  Table  5 
below  is  a  description  of  each  type  of  training  media. 

Table  5 

Description  of  Training  Media  Used  for  dL 


Type  of  Training  Media 

Description 

1. 

Classroom  training 

Training  delivered  in  the  classroom. 

2. 

Web-based  training 

Training  delivered  over  the  Internet. 

3. 

Computer-based 

training 

Training  delivered  through  a  computer 
program. 

4. 

Webinars 

Live  meetings  or  presentations  delivered  over 
the  Internet. 

5. 

Websites 

Documents  accessible  via  the  Internet. 

6. 

Videos 

Series  of  framed  images  put  together  in 
sequence,  one  after  another,  to  simulate 
motion  and  interactivity. 

7. 

Simulations 

Computer  programs  used  to  simulate  real  or 
imaginary  scenarios. 

8. 

Chat  and  discussion 
communities 

Synchronous  and/or  asynchronous  electronic 
communications  in  a  chat  room  or  online 
community. 

Each  of  these  modes  of  media  serves  a  different  purpose  for  schoolhouse 
instruction  and  dL.  However,  the  optimal  approach  to  accommodate  the  training  needs  of 
instructors  and  learners  is  to  provide  all  modes  of  media.  This  method,  while  most 
effective,  is  not  always  available.  Additionally,  as  indicated  in  Table  6,  access  to  all 
forms  of  media  may  not  always  be  the  most  cost  effective.  When  selecting  training 
media,  the  following  criteria  should  be  considered: 

•  Cognitive  processes  necessary  for  trainees  to  acquire  the  task  performance 
that  is  the  target  of  the  training  (Sugrue  &  Clark,  2000). 

•  Attributes  that  can  support  the  type,  amount,  timing,  and  control  of  methods 
selected  for  the  training  (Sugrue  &  Clark,  2000). 

•  Economical  and  convenient  set  of  media  that  possess  all  of  the  required 
attributes  (Sugrue  &  Clark,  2000). 

•  Bandwidth  connectivity.  Some  students  or  training  environments  may  be  in 
remote  locations  without  high  bandwidth,  therefore  some  content  will  not  run 
(Bersin  and  Associates,  2003). 

•  Browser  versions  and  plugins.  Students  may  have  different  browsers  and 
plugins  so  content  standards  must  be  set  which  specify  browser  version, 
plugins,  bandwidth,  memory,  and  CPU  speed  needed  (Bersin  and  Associates, 
2003). 
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Table  6 

Instructional  Value  and  Developmen  t  Architecture  of  Training  Media  (adapted  from 
Bersin  and  Associates,  2003) 


Type  of  Training 
Media 

Instructional 

Value 

Time  to 
Develop 

Cost  to 
Develop 

Cost  to 
Deploy 

Assessment 

Capability 

Web-based  training 

High 

4-20  weeks 

High 

Low 

High 

Computer-based 

training 

High 

6-20  weeks 

High 

Medium 

High 

Webinars 

Medium 

3-6  weeks 

Low 

Medium 

Low 

Websites 

Low 

1-8  weeks 

Low 

Low 

None 

Video 

High 

6-20  weeks 

High 

High 

None 

Simulations 

Very  High 

8-20  weeks 

High 

Medium 

High 

Chat  and  discussion 
communities 

Medium 

4-6  weeks 

Medium 

Medium 

None 

To  select  the  appropriate  training  media  and  delivery  methods  for  the  intended 
task  and  audience,  course  designers  should  know  the  target  audience’s  ability  to  access 
the  Internet,  consider  their  knowledge,  skills,  and  ability  to  be  trained,  know  which  skills 
are  trained  better  by  specific  media,  and  be  cognizant  of  relevant  training  research 
(Wampler,  Dyer,  Livingston,  Blackenbeckler,  Centric,  &  Dlubac,  2006).  Given  the 
limited  infrastructure  and  bandwidth  found  in  many  Army  training  environments,  the 
delivery  method  and  training  media  should  be  selected  to  work  effectively  within  these 
constraints. 

Current  and  emerging  training  technologies:  games.  A  gaming  approach  to 
training  is  certainly  not  a  new  concept  for  the  Department  of  Defense  (DOD)  community. 
The  military  has  been  incorporating  gaming  technology  into  the  curriculum  for  about  30 
years,  beginning  with  the  introduction  of  “Mech  War”  by  Jim  Dunnigan  in  the  late  1970s 
to  the  Anny  War  College.  Currently,  gaming  and  simulation  are  part  of  the  curriculum  of 
every  U.S.  war  college  and  part  of  the  operations  of  every  command  headquarters  (Herz 
&  Macedonia,  2002).  The  Marine  Corps  started  with  “Doom”  and  switched  to  “Operation 
Flashpoint.”  The  Army  included  “Spearhead”  and  other  games  into  the  Armor  School 
curriculum.  “Full  Spectrum  Command”,  developed  by  the  Institute  for  Creative 
Technologies,  is  used  in  the  Infantry  Captain’s  Career  Course  at  Fort  Benning.  “Steel 
Beasts”  is  being  used  at  the  U.S.  Military  Academy,  and  the  Navy  introduced  “Fleet 
Command”  at  the  Naval  Academy. 

Anecdotally,  training  developers  and  administrators  alike  have  supported  the 
notion  of  using  games  for  training  because  of  the  technology’s  appeal  to  this  generation 
of  Soldiers.  There  is  some  scientific  evidence  to  suggest  that  games  can  be  used  as  an 
effective  training  technology.  For  example,  evidence  suggests  that  presenting  training  in 
a  game  format  engages  learners  and  consequently  increases  the  effectiveness  of  the 
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training,  including  a  15-20  percent  improvement  of  retention  in  certain  cases  (Belanich  et 
al,  2005;  Gopher  et  al,  1994;  Prensky,  2001;). 

In  addition  to  being  familiar  and  appealing  to  many  of  this  generation  of  Soldiers, 
games  do  have  the  ability  to  recreate  some  of  the  conditions  of  today’s  network-centric 
warfare  challenges,  such  as  the  robust  networking  of  geographically  or  hierarchically 
dispersed  forces.  Massive  Multiplayer  Online  Gaming  (MMOG)  provides  trainees  with 
practice  in  fonning  and  managing  large  communities  of  participants,  developing 
communication  and  collaboration  networks,  and  re-organizing  in  order  to  handle  new 
developments.  In  fact,  in  their  review  of  games  and  simulations  studies,  Bonk  and 
Dennen  (2005)  report  that  individuals  who  are  attracted  to  MMOG  are  motivated  by 
some  of  the  key  factors  critical  to  success  in  network-centric  operations:  competition, 
collaboration,  and  potential  affiliation  with  a  global  network  of  other  players.  The  benefit 
of  MMOG  compared  to  a  single-player  gaming  approach  is  that  it  forces  participants  to 
interact  with  each  other  in  constructive  ways,  teaching  collaboration,  communications, 
and  organization  skills.  Leung  et  al  (2008)  provided  evidence  that  an  MMOG-  style 
training  event  could  be  used  to  provide  immersive,  web-based  training  to  distributed 
participants  to  teach  them  organizational  structure  and  navigation,  information 
management,  and  interagency  interaction. 

Games  also  have  the  ability  to  integrate  with  or  simulate  weapons  and  equipment 
that  require  digital  skills  and  technical  knowledge  to  operate.  Schaab  and  Dressel  (2003) 
provided  insight  on  the  type  of  technological  training  Soldiers  (in  this  case,  operators  of 
Army  Battle  Command  System  (ABCS)  at  Fort  Lewis,  Fort  Hood,  and  Fort  Bragg)  view 
as  most  productive  in  developing  expertise.  Soldiers  expressed  a  desire  and  need  for  more 
training  that  would  allow  them  to  integrate  their  knowledge  of  digital  systems  with  their 
Army  role.  They  also  expressed  willingness  to  use  gaming  technology  to  advance  their 
training.  Based  on  these  and  other  similar  findings,  changes  have  already  taken  place. 

For  example,  Fort  Huachuca  is  now  integrating  games  with  simulations  of  realistic 
missions  into  their  training  curriculum  as  much  as  possible,  to  allow  Soldiers  to  receive 
hands-on  training  in  a  realistic  setting. 

Despite  the  appeal  of  training  with  games,  there  are  still  important  scientific 
questions  and  practical  challenges  that  need  to  be  addressed.  Most  importantly,  there  is 
currently  no  consensus  on  what  types  of  skills  games  are  best  used  to  train,  nor  is  there 
convincing  scientific  evidence  demonstrating  gains  in  training  effectiveness  when  using 
games  over  other  training  technologies.  A  potential  drawback  of  using  games  for  training 
is  the  cost  of  developing  high-end  gaming  technology.  If  the  Army  were  to  invest  in 
several  gaming  development  efforts,  which  would  certainly  cost  millions  of  dollars  at 
least,  it  would  be  important  to  demonstrate  training  gains  relative  to  simpler  interactive  or 
even  didactic  technologies.  It  is  also  possible  that  some  of  the  training  time  when  using 
games  is  used  inefficiently.  For  example,  Chen  (2003)  argues  that  excessive  difficulty  in 
learning  the  interface  of  the  game  and  its  rules  may  actually  counter  its  value  as  a  tool  to 
train  the  skills  of  interest.  In  his  research,  Chen  examined  the  usefulness  of  various 
tutorial  techniques  used  in  current  commercial  off  the  shelf  (COTS)  computer  games  in 
helping  novice  players  reach  an  acceptable  level  of  skill,  which  enabled  them  to  play  the 
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game  effectively.  In  addition,  he  investigated  how  instructional  media  (i.e.,  game  tips  and 
computer-based  tutorials)  affect  the  acquisition  of  basic  computer  game-playing  skills. 
The  results  of  the  research  demonstrated  that  depending  on  the  game  being  learned, 
different  instructional  techniques  should  be  utilized.  For  example,  games  that  involve 
maneuvering  and  actions  would  require  different  instructions  than  games  with 
predominantly  cognitive  components.  This  is  just  one  example  of  the  type  of  research 
that  is  required  to  fully  determine  when  and  how  this  promising  technology  can  live  up  to 
its  training  potential. 

Incorporating  SMEs  into  training  development.  In  order  to  ensure  the  selection 
of  the  most  appropriate  training  methods  and  media  in  dL  course  design,  it  is  critical  to 
receive  input  from  a  variety  of  contributors.  There  is  no  one  person  that  possesses  all  the 
skills  to  develop  and  deliver  a  distance-learning  course.  Therefore,  SMEs,  instructional 
designers,  and  media  specialists  are  essential  for  every  team  (Simonson,  2007).  SMEs 
play  an  important  role  in  the  instructional  system  decision  process.  They  provide  input  on 
perspectives  of  different  users  and  skill  levels  as  well  as  knowledge  about  lessons  learned 
from  real-world  operations.  All  this  infonnation  could  and  should  be  incorporated  in  the 
design  of  dL  courses.  In  addition,  in  the  Army  context,  SMEs  ensure  that  current 
doctrinal  information  and  procedures  are  included  in  the  training  design,  and  correct 
military  terminology  is  being  used.  They  may  also  suggest  potential  uses  and  benefits  of 
the  system  that  might  not  be  recognized  by  the  course  designer.  Keeping  in  mind  the 
benefits  that  SMEs  could  bring  to  the  design  process,  collaboration  between  them  and 
instructional  designers  is  essential  to  the  development  of  effective  training. 

Typically,  SMEs  are  experts  in  a  certain  training  topic,  whereas  course  designers 
are  experts  in  training  development.  Most  likely,  it  would  be  difficult  for  a  SME  to  play 
the  role  of  a  course  designer  and  develop  dL  training  modules.  However,  there  are 
authoring  tools  such  as  Adobe®  Captivate®,  Camtasia®,  Articulate®,  and  various 
Learning  Content  Management  Systems  that  may  help  SMEs  in  the  development  of  dL 
modules.  In  combination,  these  products  provide  tools  to  create  SCORM-compliant  e- 
learning  modules,  interactive  content,  videos,  assessments,  feedback,  and  surveys.  Table 
7  displays  features  the  authoring  software  offers. 
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Table  7 

Features  of  authoring  software. 


Type  of 
Authoring 
Software 

Overview 

Interactive 

content 

Videos 

Assessments 

Surveys 

SCORM- 

compliant 

Learning 

Content 

Management 

Systems 

Provide  rapid 
creation  of  e- 
learning  modules 
and  ability  to  reuse 
content  for  other 

X 

X 

X 

courses 

Adobe 

Captivate 

Provides  rapid 
creation  of 
simulations, 
scenario-based 
training,  and 
quizzes  without 
programming 
knowledge  or 
multimedia  skills 

X 

X 

X 

Camtasia 

Studio 

Records  screen 
captures  and  audio 
for  video  and 
scenario 
development 

X 

X 

Articulate 

Provides  rapid 
creation  of  e- 
learning  modules 
from  PowerPoint, 

X 

X 

X 

X 

X 

quizzes, 

assessments,  and 

surveys 

These  tools  can  serve  as  an  interface  between  SMEs  and  course  designers  to  help 
involve  SMEs  in  the  development  process  by  transforming  the  knowledge  content  from 
SMEs  into  robust  and  interactive  dL  modules.  However,  to  effectively  integrate  SMEs 
into  the  design  process,  it  is  necessary  to  teach  them  how  to  use  the  authoring  tool.  This 
could  be  done  through  one-on-one  training  with  a  course  designer,  self-paced  web-based 
course,  online  seminars,  or  classroom  training.  Once  SMEs  are  trained,  the  course 
designer  could  choose  from  two  approaches  to  developing  the  training  program:  the 
individual  approach  or  the  team  approach.  The  individual  approach  provides  SMEs  the 
authority  to  capture  content  for  dL  courses  and  the  course  designers  to  edit  and  enhance 
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that  content.  Alternatively,  the  team  process  requires  SMEs  and  course  designers  to  work 
together  on  creation  of  storyboards  and  scripts  prior  to  the  capture  of  content  by  SMEs. 
With  either  process,  utilizing  authoring  tools  improves  the  cycle  time  for  developing 
training  program  and  reduces  the  number  of  iterations  between  SMEs  and  course 
designers. 

With  either  process,  course  designers  and  developers  should  be  aware  that  many 
times  SMEs  may  inadvertently  omit  information  which  to  them  may  be  incidental  or  even 
automated  (Clark,  Feldon,  van  Merrienboer,  Yates,  and  Early,  2007).  The  example 
authoring  tools  given  above  do  not  avoid  the  need  for  some  sort  of  knowledge  elicitation 
or  validation  function  that  will  mediate  this  problem. 

Considerations  from  learning  science  and  technology.  Any  appeal  to 
learning  science  and  technology  begs  the  question  of  whether  Army  institutional  training 
can  be  made  significantly  more  effective.  The  Army  has  been  in  the  business  of  training 
Soldiers  for  decades  using,  in  general,  much  the  same  instructor-led  lecture  training 
methods  that  are  traditionally  associated  with  public  education.  The  issue  arises:  is  there 
evidence  to  support  the  assumption  that  the  effects  of  this  training  can  be  improved,  and, 
if  so,  even  more  importantly,  is  the  improvement  of  a  significance  that  would  warrant 
changing  Army  training?  This  is  not  a  trivial  question:  in  a  similar  learning  environment, 
Dubin  and  Taveggia,  as  late  as  1968,  concluded,  after  an  extensive  re-analysis  of  more 
than  100  studies  of  college  teaching  methods,  that  there  was  no  compelling  empirical 
evidence  to  prefer  one  over  another. 

Although  there  have  been  since  Dubin  and  Taveggia ’s  caution  numerous 
reassuring  quantitative  observations  of  the  results  of  numerous  modifications  to 
conventional  instruction,  one  of  the  most  cogent  and  well-known  was  made  by  Bloom 
(1984).  Bloom  and  his  colleagues  investigated  the  effects  of  conventional  classroom 
instruction  versus  tutoring  upon  achievement  measures  for  a  given  subject  matter.  On  the 
final  achievement  measures,  the  average  tutored  student’s  performance  was  about  two 
standard  deviations  above  that  of  the  average  conventionally  instructed  student.  In 
reporting  this  and  other  similar  supporting  studies,  Bloom  noted  that  the  results  indicate 
that  (a)  alternatives  to  conventional  instruction  can  produce  an  effect  as  much  as  “2 
sigma”  (two  standard  deviations)  above  conventional  instructor-led  classroom  instruction 
and  (b)  this  effect  can  be  achieved  by  the  average  student.  The  general  implications  for 
Anny  institutional  training  parallel  Bloom’s  notes:  (a)  given  the  potential  pay-off  in 
effectiveness,  it  is  reasonable  for  the  Army  to  investigate  alternatives  to  conventional 
instructional  methods  and  (b)  it  is  reasonable  to  assume  that  the  effectiveness  gains  will 
apply  across  all  Soldiers. 

It  goes  without  saying  that  Army  institutional  training  is  not  resourced  to  provide 
a  personal  tutor  for  each  Soldier  -  any  changes  to  training  must,  at  least  initially,  fit 
reasonably  within  the  current  training  constraints  and  environment.  Thus,  although  the 
Army  can,  in  pursuit  of  something  similar  to  Bloom’s  “2  sigma”  effect,  turn  to  learning 
science  for  solutions,  the  application  of  that  science  is  perforce  restricted  to  and 
constrained  by  the  Army’s  institutional  training  environment.  For  example,  learning 
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science  principles  specific  to  public  school  grades  K  through  6  may  be  of  questionable 
applicability  to  the  Army  school  population  of  adult  learners  aged  18  and  older. 

Similarly,  implementation  of  some  learning  science  principle  involving  continuous 
monitoring  of  some  physiological  variable,  such  as  eye  movements,  would  be  of  no  direct 
use  in  the  Anny  institutional  training  environment.  In  the  considerations  below,  a 
conscious  effort  has  been  made  to  present  findings  against  the  background  of  the  Anny 
institutional  training  environment. 

Summaries  or  snapshots  of  the  status  of  learning  science  appear  from  time  to  time 
for  education  and  learning  in  the  public  school  system  (e.g.,  Bransford  et  ah,  2000)  and 
also  within  the  military  context  (e.g.,  Driskell  et  ah,  1995;  Cumow  et  al.,  2006).  From 
these  and  similar  works,  it  becomes  evident  that  within  the  science  of  learning,  there  are 
several  principles  of  learning  about  which  nearly  all  researchers  are  in  agreement,  and 
there  are  several  theories  of  learning  about  which  there  is  considerably  less  agreement. 

Merrill  (2002)  makes  the  assertion  that,  theoretical  viewpoint  notwithstanding, 
there  are  principles  of  learning  that  obtain  “regardless  of  the  instructional  programs  or 
practices  involved.”  Merrill  claims  these  principles  of  learning  have  been  reliably 
demonstrated  in  learning  research  literature  to  facilitate  learning,  and  further 
hypothesizes  that  the  effectiveness  of  any  approach  to  learning  will  be  affected  to  the 
extent  that  the  approach  incorporates  all  these  principles.  Merrill’s  principles  are 
outlined  in  Table  8  below.  As  might  be  expected,  most  learning  theorists  agree  with  all 
the  principles;  the  disagreement  arises  in  their  interpretation  and  implementation. 

Table  8 

Principles  of  Learning  (adapted  from  Merrill,  2002) 


Problem 

The  learner  must  be  engaged  in  solving  real-world  problems 
associated  with  the  learning  goals.  The  problems  must  have  a 
range  of  difficulty.  The  learner  must  explicitly  compare 
problems  one  with  another 

Activation 

The  learner  must  activate  applicable  knowledge  from  past 
experience  for  use  as  a  foundation  for  the  new  knowledge 

Demonstration 

The  learner  must  be  shown  rather  than  told  and  must  be  shown 
multiple  representations  of  the  problem.  Demonstrations  must 
be  relevant  to  the  learning  goals  and  must  direct  the  learner  to 
relevant  information.  The  learner  must  explicitly  compare 
alternative  representations. 

Application 

The  learner  must  use  the  new  knowledge  to  solve  problems 
consistent  with  the  learning  goal.  The  learner  must  be  guided 
by  coaching  that  is  gradually  withdrawn  and  must  be  shown 
how  to  detect  and  correct  errors. 

Integration 

The  learner  must  verbalize  and  demonstrate  the  new 
knowledge  or  skill.  The  learner  must  generate  new,  personal 
ways  to  use  the  new  knowledge  or  skill. 
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Although  he  purports  to  be  more  pragmatic  than  theoretical  (personal  communication, 
2008),  Merrill  maintains  that,  to  be  successful,  any  theory  of  learning  must  account  for  all 
five  principles. 

Merrill’s  pragmatism  is  no  doubt  driven  in  part  by  there  being  no  single 
consensual  “Grand  Theory”  of  learning  (cf.  Duchastel,  1998;  Jonassen,  2003).  However, 
there  has  been  through  the  20th  century  a  progression  of  general  paradigms  of  learning 
that  can  be  characterized  as  roughly  paralleling  that  of  the  progression  of  theories  and 
models  of  general  psychology.  Mayer  in  1996  outlined  this  progression  as  three 
“metaphors”: 

•  1900s -  1950s:  Behaviorism 

•  1960s-  1970s:  Information  processing 

•  1980s  -  present:  Constructivism 

with  information  processing  and  constructivism  both  arising  from  cognitive  psychology. 
Others  (e.g.  Clark  in  Quinkert  et  al.,  2007)  propose  similar  categorizations,  although  with 
some  variation  in  category  labels.  An  abbreviated  characterization  of  the  three  appears  in 
Table  9  below.  Mayer  (1996)  was  careful  to  point  out  that  the  progression  is  cumulative 
in  the  sense  that  no  earlier  metaphor  has  been  completely  replaced  by  a  later  metaphor. 

As  Kirschner,  Sweller,  and  Clark  argue  as  late  as  2006,  there  is  still  lack  of  consensus 
within  the  field  of  learning  theory  that  “constructivism”  has  displaced  “information 
processing”  as  a  more  effective  model  of  learning  —  Kirschner  et  al.  forcefully  argue  the 
case  that  “constructivism”  is  still  at  best  an  unproven  alternative  contemporaneous  with 
“information  processing.” 

Kirschner  et  al.’s  2006  paper  echoes  the  discussions  currently  taking  place  in 
learning  theory.  Arguments,  theoretical  and  empirical,  abound  as  to  the  relative 
effectiveness  of  the  three,  with  the  majority  of  discussion  addressing  the  merits  of  an 
information  processing  viewpoint  compared  to  a  constructivist  viewpoint.  As 
background  to  an  understanding  of  the  current  state  of  learning  science,  some  of  the 
specific  tenets  of  the  information  processing  and  the  constructivist  viewpoints  are  given 
in  Table  9  below. 
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Table  9 

Characterization  of  Mayer’s  (1996)  learning  “metaphors” 


Behaviorism 

Information  Processing 

Constructivism 

Philosophy 

“Learning”  is  manifested  in 
specific  recurring  behavior  under 
specific  conditions.  Recurrence 
of  behavior  depends  on  the 
quality  of  its  immediate  conse¬ 
quences. 

Meaning/knowledge  is  present  in 
and  extracted  from  the 
environment;  it  can  be 
selectively  stored  in  memory 
(learned)  and  retrieved. 

Meaning/knowledge  is  cumu¬ 
latively  constructed  by  the 
individual  as  result  of  actively 
making  sense  of  problems  within 
the  environment 

Instruction  function 

Supply  consequence  when 
learner  emits  specific  behavior 
under  specific  conditions 

Actively  guide  student’s 
processing  of  well- structured 
information 

Facilitate  student’s  construction 
of  meaning/knowledge  as 
necessary  during  sense-making 
activity 

Learner  role 

Strengthen  association  between 
specific  behavior  and  specific 
conditions 

Actively  select  and  encode 
information  for  subsequent 
retrieval 

Actively  construct  and  assimilate 
individual  version  of  information 
relative  to  problems 

Learning  outcome 

Specific  behavior  reliably  recurs 
in  response  to  appropriate 
conditions 

Learner  can  at  will  retrieve  the 
learned  information. 

Learner  can  autonomously  make 
sense  of  problems 

Learning  assessment 

Observe  likelihood  of  learner 
emitting  specific  behavior  in 
response  to  appropriate 
conditions 

Observe  degree  to  which  learner 
accurately  retrieves  information 
presented  in  instruction 

Observe  ability  to  make  sense  of 
problems  similar  to  those 
encountered  during  learning 
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The  information  processing  “ metaphor  ”  .  As  mentioned  above,  the  information 
processing  point  of  view  is  based  in  cognitive  psychology.  The  structure  typically 
espoused  is  based  on  Baddeley’s  (e.g.,  1992)  elaboration  of  Atkinson  and  Shiffrin’s  1968 
sensory  store  -  short  term  memory  -  long  term  memory  model  and  on  Miller’s  1956 
work  in  short  term  memory  capacity  and  unit  of  processing.  The  processes  typically 
espoused  involve: 

1 .  Attending  to  and  selecting  relevant  information  from  sensory  memory  into 
short  term  memory. 

2.  Organizing  the  information  in  short  term  memory  into  some  sort  of 
meaningful  whole 

3.  Connecting  the  “meaningful  whole”  in  short  term  memory  to  some  knowledge 
structure  already  existing  in  long  term  memory. 

Short  term  memory  processing.  Miller’s  capacity  results  are  applied  to  the  short 
term  memory  construct:  it  is  posited  that  short  term  memory  has  a  capacity  of,  at  best,  an 
average  of  seven  items,  or,  at  worst,  an  average  as  low  as  four  items  (Cowan,  2001;  cf. 
Unsworth  and  Engle,  2007).  Items  in  short  term  memory  may  be  selected  from  sensory 
memory  and  may  also  be  retrieved  from  long  term  memory.  Because  these  items  of 
information  must  be  held  in  short  term  memory  for  some  amount  of  time  as  they  are 
being  organized,  it  becomes  critical  that  the  processing  capacity  of  short  term  memory 
not  be  overloaded  by  either  (a)  the  rate  at  which  items  are  presented  via  sensory  memory 
or  (b)  the  number  of  items  that  must  be  simultaneously  held  while  being  organized  in 
short  term  memory  .  The  learner  is  assumed  to  control  the  short  term  memory  items 
drawn  from  long  term  memory,  but  if  the  presented  load  from  sensory  memory  exceeds 
short  term  memory  processing  capacity,  then  some  items  are  in  some  manner  “missed” 
and  learning  becomes  less  effective. 

Since  short  term  memory  capacity  is  perceived  as  a  bottleneck  to  the  model, 
considerable  attention  (e.g.,  Sweller,  van  Merrienboer,  and  Paas,  1988)  has  been  devoted 
to  that  aspect  of  the  information  processing  model.  All  information  arriving  at  sensory 
memory  must  be  in  some  manner  sorted  by  the  learner  so  that  the  relevant  may  be 
attended  to  and  submitted  to  some  sort  of  information  organizing  activity.  Researchers 
(e.g.,  Sweller,  1994)  have  classified  the  presented  cognitive  information  load  into  three 
types:  intrinsic,  germane,  and  extraneous  load.  Intrinsic  load  refers  to  the  load  presented 
by  the  information  that  is  to  be  learned.  Germane  load  refers  to  the  load  presented  by  the 
information  that  serves  to  assist  in  assimilating  the  intrinsic  load.  And  extraneous  load 
refers  to  the  load  presented  by  the  information  that  is  irrelevant  to  the  information  to  be 
learned. 

Note  that  the  intrinsic/germane/extraneous  distinction  is  opaque  to  the  naive 
learner;  absent  any  elucidating  instructions  from  the  external  environment,  all  incoming 
information  is  potentially  intrinsic  to  the  learner.  This  implies  that  to  the  extent  the 
instructor  or  training  designer  can,  within  the  limits  of  short  term  memory  capacity,  direct 
the  learner’s  attention  to  the  appropriate  (intrinsic)  information,  can  optimize  the  germane 
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information,  and  can  minimize  extraneous  load,  efficiency  of  the  organizing  function  in 
short  term  memory  should  increase. 

Long  term  memory  processing.  Similarly,  the  existing  long  term  memory 
knowledge  structure  with  which  the  learned  infonnation  is  to  be  integrated  may  not  be 
obvious  to  the  learner.  This  implies  that  to  the  extent  the  instructor  can  specify  the  long 
tenn  memory  knowledge  to  which  the  newly  learned  information  is  related,  efficiency  of 
retrieval  of  information  from  long  term  memory  into  short  tenn  memory  and  the  transfer 
from  short  tenn  memory  to  long  term  memory  should  increase. 

Miller’s  (1956)  unit  of  processing  for  short  term  memory  is  applied  to  the 
knowledge  structures  in  long  term  memory.  Similar  to  Miller’s  promotion  of  the  idea  of 
“chunking”  in  short  term  memory,  the  infonnation  theory  approach  appeals  to  the 
construct  of  a  “schema”  (cf.  Bartlett,  1932)  in  long  term  memory.  Learners  in  some 
meaningful  way  combine  related  items  into  a  schema  which  then  becomes  an  information 
item  itself  and  available  to  short  term  memory.  For  example,  by  combining  appropriate 
elements  of  information,  learners  may  create  a  schema  for  the  “quadratic  formula,”  and 
then,  in  turn,  use  the  quadratic  formula  schema  as  an  information  element  in  solving 
general  quadratic  equations. 

At  a  high  level,  the  more  generalized  course  of  learning  is  characterized  as 

1.  Subject  to  short  term  memory  capacity  limits,  information  items  from 
sensory  store  and/or  schemata  from  long  term  memory  are  selected  into 
short  tenn  memory. 

2.  The  learner  organizes  the  information  into  some  meaningful  whole. 

3.  The  resulting  product,  a  new  schema  or  a  modified  schema,  is  associated 
with  some  knowledge  structure  already  in  long  term  memory. 

Considerations  relative  to  efficiency.  From  this  general  architectural  and  process 
model  of  learning,  several  principles  enhancing  learning  effectiveness  and  efficiency  can 
be  derived: 


•  Organize  learning  to  progress  from  simple  elements  to  more  complex 
elements 

•  Sequence  learning  in  a  logical  order 

•  Avoid  introducing  too  many  new  items  at  once. 

•  Use  discretion  in  introducing  explanatory  or  supporting  material 

•  Allow  the  learner  opportunity  for  reflection  during  instruction 

•  Point  out  relationships  between  the  to-be-learned  material  and  material 
already  learned 

•  Accentuate  the  new  items  that  are  to  be  attended  to  during  learning 


2  Generic  form  is: 


-b±  sjb1  -4  ac 

x  = - 

2  a 


We  will  return  to  this  result  later  on  in  a  following  discussion. 
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•  Minimize  extraneous  distractions  during  learning 

These  principles  remain  the  foundation  of  most  information  processing  based 
approaches  to  learning.  In  addition  to  these  principles,  current  approaches  have 
accommodated  additional  features  not  directly  derivable  from  the  information  processing 
model,  such  as  casting  learning  in  the  context  of  problem  solving  and  allowing  for  the 
cumulative  effects  of  learning  task  repetition.  Examples  of  current  information 
processing  approaches  include: 

•  ISD/ADDIE  (instructional  systems  design/analyze-design-develop-implement- 
evaluate)  (e.g.,  Gagne  and  Briggs,  1973;  Branson,  1975) 

•  GEL  (guided  experiential  learning)  (Clark,  2004) 

•  4C/ID  (four  component  instructional  design)  (e.g.,  van  Merrienboer,  Clark,  and 
de  Croock,  2002) 

•  Army  SAT  (systems  approach  to  training)  (U.S  Army  Training  &  Doctrine 
Command,  1999) 

All  these  approaches  adopt  a  position  of  all  relevant  information  or  knowledge  being 
marshaled  up  and  transferred  or  transmitted  to  the  learner  from  the  external  environment 
via  instructor,  text,  etc. 

The  constructivism  “ metaphor .  ”  While  the  infonnation  processing  approach 
derives  from  a  characterization  of  the  process  by  which  information  or  knowledge  is 
input,  the  constructivist  approach  derives  from  a  characterization  of  the  internal 
structuring  of  knowledge.  Constructivism  holds  that  the  learner,  through  purposeful, 
active  experience  with  the  environment,  constructs  his  or  her  own  personal  representation 
of  the  to-be-learned  material.  This  new  construction  is  necessarily  built  upon  and 
mutually  supportive  with  the  learner’s  own  existing  knowledge  structures.  The 
implication  is,  since  each  learner’s  existing  knowledge  structures  are  different,  that, 
across  learners,  each  learner  constructs  his  or  her  own  personal  version  of  the  to-be- 
learned  knowledge  (cf.  von  Glasersfeld,  1984).  This  process  of  cognitive  construction 
can  also  be  termed  as  one  of  “sense  making”  or  cognitive  dialectics.  That  is,  the  learning 
process  is  the  process  of  cognitively  assimilating  and  making  personal  sense  of  the  to-be- 
learned  material  and  of  reconciling  it  with  existing  personal  knowledge. 

From  the  constructivist  point  of  view,  learning  inescapably  occurs  within  and  is 
influenced  by  the  social  environment  (von  Glasersfeld,  1997);  from  this  it  can  be  inferred 
that,  for  the  current  Anny  institutional  environment,  learning  is  mediated  not  only  by  the 
instructor,  but  also  by  classroom  peers.  But,  even  though  it  is  grounded  in  the  social 
environment,  it  is,  at  the  individual  learner  level,  purposeful.  The  learner  may  engage  in 
the  learning  process  merely  to  pass  an  end-of-course  exam,  or  the  learner  may  engage  in 
the  learning  process  to  gain  an  understanding  of  some  concept.  In  either  case,  the  learner 
has  a  purpose,  and,  more  importantly,  the  purpose  plays  a  determining  role  in  how  the 
knowledge  is  assimilated.  The  purpose  of  learning  can  be  facilitated  by  its  being  cast  in 
terms  of  being  based  or  situated  in  some  common  referent  or  context  (e.g.  Cognition  and 
Technology  Group  at  Vanderbilt,  1993)  or  in  terms  of  some  relevant  to-be-solved 
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problem  or  task  (e.g.,  Savery  and  Duffy,  1995).  In  the  extreme  constructivist  point  of 
view  (cf.  von  Glasersfeld,  1984),  the  learner  would  be  provided  at  most  just  the  initial 
context  or  problem  conditions  and  access  to  information  relative  to  the  context  or  to 
constructing  a  solution  to  the  problem.  Within  this  environment,  the  learner  is  faced  with 
“making  sense”  of  the  situation  or  discovering  solutions.  As  result  of  this  increased 
cognitive  activity,  the  learner  is  thought  to  gain  a  deeper,  more  elaborated  understanding 
of  the  problem  area.  Ostensibly,  as  the  product  of  such  an  instructional  environment,  a 
student  faced  with  solving  the  “quadratic  formula”  (footnote  1,  page  45)  for  the  constant 
value  c  would  have  the  deeper  understanding  that,  since  the  quadratic  formula  is  derived 
from  the  quadratic  equation  ax2  +  bx  +  c  =  0,  the  solution  can  be  directly  found  as  c  =  - 
ax'  -  bx,  thereby  bypassing  the  more  obvious  but  algebraically  torturous  route  of  deriving 
c’s  value  by  working  directly  from  the  quadratic  formula.  Within  the  pure  constructivist 
instructional  environment,  because  the  learner  is  to  construct  his  or  her  own  personal 
learning  via  interacting  with  the  situation  in  a  real  world  context,  the  instructor’s  role 
changes  from  one  of  direct  instruction  to  one  of  at  best  indirect  monitoring  and  mentoring 
of  the  learner’s  progress. 

As  Mayer  (2004)  has  pointed  out,  there  is  scant  empirical  support  for  the 
effectiveness  of  such  an  extreme  stance  -  “pure”  constructivism  is  generally  an 
inefficient  approach  for  achieving  specific  training  outcomes.  As  an  example,  Dolmans, 
Gijselaers,  Schmidt,  and  van  der  Meer  (1993)  found  in  one  case  that  learner-generated 
issues  failed  to  address  as  much  as  30%  of  the  instructor’s  intended  objectives.  The 
conclusion  drawn  is  that,  although  a  constructivist  approach  can  eventually  be  effective, 
without  considerable  intervention  it  will  never  be  efficient. 

Taking  a  different  perspective,  several  (e.g.,  Airasian  and  Walsh,  1997;  Bransford 
et  ah,  2000;  Kirschner,  2009;  Jonassen,  2006)  have  taken  pains  to  point  out  that  there  is  a 
distinction  to  be  made  between,  on  the  one  hand,  a  constructivist  epistemology 
characterizing  what  we  know  and  how  we  know  it  and,  on  the  other,  the  pedagogical 
methods  that  comprise  the  strategies  and  techniques  used  to  promote  construction  of 
knowledge.  That  is,  from  a  practical  point  of  view,  constructivism  in  and  of  itself  does 
not  absolutely  pedagogically  proscribe  any  assistance  being  given  to  the  learner. 

Scaffolding.  This  perspective  leads  to  one  of  the  more  vociferously  pursued 
current  issues  amongst  learning  theorists.  At  issue  is  the  amount  of  assistance  to  be  given 
the  learner  and,  much  more  importantly,  the  nature  of  that  assistance.  While  the 
information  processing  approach  prescribes  directly  providing  unsolicited  proactive 
assistance  to  guide  the  learner  in  constructing  knowledge,  the  constructivist  approach 
would  say  to  indirectly  provide  reactive  assistance  only  as  needed  by  the  learner  during 
knowledge  construction.  Typically,  constructivists  build  on  Vygotsky’s  (1978)  idea  of  a 
personal  zone  of  proximal  development.  This  zone  is  defined  as  the  space  or  difference 
between  the  learner’s  current  problem  solving  ability  (actual  development  level)  and  the 
learner’s  problem  solving  ability  under  instructional  guidance  or  collaboration  with  peers. 
As  learning  progresses,  the  locus  of  the  zone  changes  -  “the  zone  of  proximal 
development  today  will  be  the  actual  developmental  level  tomorrow”  (Vygotsky,  1978,  p. 
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87).  To  the  extent  that  the  learner  can  maintain  position  at  the  edge  of  this  zone  and  can 
access  external  guidance  as  needed,  learning  will  be  efficiently  effected. 

Maintaining  the  learner  at  or  near  the  edge  of  Vygotsky’s  zone  of  proximal 
development  is  implemented  via  scaffolding.  First  named  by  Wood,  Bruner,  and  Ross  in 
1976,  the  concept  of  scaffolding  was  originally  developed  within  the  environment  of 
tutors  instructing  pre-school  children.  Wood  et  al.  noted  that  the  tutors  provided,  as 
needed,  focused,  individualized  support  to  the  learners,  and,  when  the  support  was  no 
longer  needed,  abandoned  it,  similar  to  temporary  scaffolding  used  in  support  of 
constructing  a  building.  Scaffolding  was  characterized  as  any  action  which  “recruit[s]  . . . 
attention,  reduces  degrees  of  freedom  in  the  task  to  manageable  limits,  maintains 
‘direction’  in  the  problem  solving,  marks  criticial  [sic]  features,  controls  frustration  and 
demonstrates  solutions  when  the  learner  can  recognize  them”  (p.  99). 

With  the  acceptance  of  scaffolding  into  constructivist  methods,  constructivism 
began  to  proliferate  a  variety  of  pedagogical  models,  all  at  least  loosely  associated  with 
the  tenet  that  learners  learn  most  effectively  within  the  context  of  a  to-be-solved  problem 
or  task.  TRADOC  Pamphlet  350-70-4  (in  preparation)  lists  a  spanning  set  of  these 
constructivist  models  (p.  3 1)  and  subsumes  them  under  the  term  problem  centered 
instruction  or  PCI.  For  this  paper,  “PCI”  will  be  used  as  short-hand  for  constructivist 
models. 

PCI  learning  theorists  and  practitioners  were  quick  to  generalize  the  concept  of 
scaffolding  from  pre-schoolers  to  learners  in  general,  from  tutor-student  interactions  to 
teacher-class  interactions,  and  from  individualized  learning  support  to  group-level 
learning  support.  Also,  with  the  introduction  of  automated  teaching  systems  (e.g., 
intelligent  tutoring  systems),  interest  turned  to  scaffolding  delivered  via  technology.  One 
recent  popular  educational  psychology  resource  finishes  up  a  defining  characterization  of 
scaffolding  with  “. .  .or  anything  else  that  allows  the  student  to  grow  in  independence  as  a 
learner”  (Woolfolk,  2007,  p.  48).  As  Davis  and  Miyake  (2004)  point  out,  conceptions  of 
and  investigations  of  scaffolding  have  become  divergent  to  the  extent  that  communi¬ 
cation  among  different  researchers  within  the  field  can  become  problematical. 

Woolfolk’s  (2007)  “. . .  or  anything  else. . .”  could  present  a  daunting  array  of  scaffolding 
material  for  consideration. 

Scaffolding,  in  the  PCI  model,  is  the  most  effective  and,  hence  for  this  paper,  one 
of  the  most  important  vehicles  by  which  the  learner  interacts  with  instruction. 
Accordingly,  for  purposes  of  development  of  this  paper,  it  is  necessary  that  it  be 
explicitly  defined  and  differentiated  from  general  instructional  activity.  Drawing  on 
Vygotsky’s  model  and  terminology,  for  this  paper  scaffolding  will  be  considered  to  be 
the  reactive  employment  of  instructional  activity  appropriate  to  the  learner’s  current  zone 
of  proximal  development.  Under  this  definition,  scaffolding  (a)  is  seen  more  as  a  process 
than  an  object  or  instructional  medium,  (b)  is  employed  in  a  manner  tailored  to  the 
learner’s  current  zone  of  proximal  development,  and  (c)  changes  as  the  learner’s  zone 
changes.  Thus,  whether  the  scaffolding  action  consists  of  hints,  worked  examples,  or 
breaking  the  problem  into  smaller  pieces  (just  a  sampling  of  objects  or  media  which  have 
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been  offered  as  scaffolding  instances)  is  not  a  question  of  interest.  The  real  interest  lies 
in  (a)  whether  the  action  is  appropriate  to  the  stage  at  which  the  learner  may  be  in 
constructing  knowledge,  and  (b)  whether  the  action  is  discontinued  when  no  longer 
needed  or  no  longer  appropriate. 

The  discontinuation,  or  “fading,”  of  scaffolding  in  PCI  once  it  is  no  longer  needed 
by  the  learner  has  received  considerable  attention  (cf.  Davis  and  Miyake,  2004).  At 
debate  are  issues  such  as:  What  does  it  mean  to  “gradually”  fade  scaffolding?  If  an 
instructional  object  is  not  faded,  is  it  truly  scaffolding?  For  purposes  of  this  paper,  if 
scaffolding  is  understood  to  consist  of  employment  of  appropriate  instructional  activity, 
then  rate  of  fading  of  the  object  or  removal  of  the  object  become  incidental;  the  issue  is 
whether  the  current  instructional  activity  is  appropriate  to  the  learner’s  current  zone  of 
proximal  development. 

If  scaffolding  is  cast  in  terms  of  its  appropriateness  to  the  individual  learner’s 
current  situation,  then  an  obvious  issue  arises  in  the  classroom  environment:  if  all 
learners  are  building  their  own  personal  knowledge  structures  at  their  own  rate  of 
learning,  then  how  can  the  instructor  match  scaffolding  across  a  class  full  of  learners  who 
at  any  single  point  in  time  presumably  have  different  zones  of  proximal  development?  A 
parallel  issue  arises  in  blended  learning  situations  using  single-student  automated 
teaching  systems  (e.g.,  intelligent  tutoring  systems):  does  the  teaching  system  have  the 
“intelligence”  to  determine  what  scaffolding  is  appropriate  to  the  student  and  when  that 
scaffolding  is  appropriate? 

In  the  case  of  classroom  scaffolding,  as  part  of  the  social  aspects  of  knowledge 
construction,  classmates  are  assumed  to  engage  in  something  approximating  peer 
scaffolding.  Vygotsky  (1978)  initially  posited  the  possibility  of  peer  scaffolding,  and  Lai 
and  Law  (2006)  and  Fawcett  and  Garton  (2005)  have  found  evidence  for  the  efficacy  of 
peer  scaffolding. 

Fawcett  and  Garton’s  (2005)  results  can  be  taken  to  indicate  that  peer  scaffolding 
can  result,  at  some  level,  in  a  “leveling”  of  a  class’s  learning:  absent  instructor 
intervention,  low  scoring  students  who  verbally  interacted  with  other  low  scoring 
students  showed  no  change  on  a  post-test,  whereas  low  scoring  students  who  verbally 
interacted  with  high  scoring  students  subsequently  increased  their  post-test  scores.  It 
should  be  noted  that  Fawcett  and  Garton’s  results  were  obtained  from  a  contrived 
experimental  setting;  the  real  world  conditions  that  promote  peer-to-peer  scaffolding  and 
the  optimal  conditions  for  its  efficacy  (cf.  Surowiecki,  2004)  deserve  additional 
investigation. 

However,  the  results  do  add  credence  to  an  assumption  that  is  many  times  implicit 
in  PCI  classrooms:  members  of  a  group  of  learners  can  be  considered  to  progress  as  a 
group,  and,  in  addition  to  personal  scaffolding,  the  instructor  can  successfully  invoke 
scaffolding  at  the  group  level.  But,  in  turn,  this  leads  to  issues  regarding  group  size  and 
composition.  Army  Regulation  350-1  (Department  of  the  Army,  2007)  assumes  group 
leader  to  student  ratios  ranging  from  1:6  to  1:16  (para  3-5. a(2)).  Wyss,  Tai,  and  Sadler 
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(2007)  found  indications  that  when  group  size  exceeds  10,  effect  of  training  may 
decrease,  while  Lou  et  al.  (1996)  found  group  sizes  of  3-4  to  be  more  effective  than  group 
sizes  of  5-6.  The  issue  of  group  size  remains  open;  there  are  strong  indications  of  an 
interaction  between  group  size  and  different  subject  matters  (Raimondo,  Esposito,  & 
Gershenberg,  1990)  -  larger  groups  may  be  effective  for  learning  lower  level  cognitive 
skills  while  smaller  groups  may  be  more  effective  for  learning  higher  level  cognitive 
skills.  An  associated  issue  which  does  not  appear  to  have  been  addressed  is  that  of 
characterizing  the  number  of  small  groups  that  an  instructor  can  effectively  and 
efficiently  supervise  at  one  time. 

Problem  generation.  Another  consideration  associated  with  subject  matter  is  the 
composition  of  the  problems,  issues,  or  questions  that  an  instructor  may  use  to  form  the 
impetus  for  a  PCI  training  event.  Dabbagh  et  al.  (2000)  characterize  the  composition  as 
being  “a  collection  of  carefully  crafted,  ill-structured  problems  that  reveal  the  underlying 
principles  and  concepts  of  a  knowledge  domain  through  descriptions  of  real  life  events 
and  experiences  and  serve  as  the  stimulus  and  focus  for  student  activity  (p.  63).” 
Although  there  appear  to  be  no  generally  accepted  problem  generation  rules,  Jonassen 
and  Hung  (2008),  summarizing  across  several  sources,  posit  the  following  principles  for 
designing  a  PCI  problem.  The  problem  should  be: 

•  open-ended  and  moderately  ill-structured 

•  complex  to  the  degree  that  it 

o  is  challenging  and  motivating,  and  engages  the  learner’s  interests 
o  provides  learners  opportunities  to  treat  the  problem  from  multiple 
points  of  view 

o  is  integrated  with  learners’  prior  knowledge 
o  is  appropriate  to  learners’  cognitive  development  and  readiness 

•  contextualized  to  the  learners’  target  workplace 

Hung  (2006)  goes  beyond  these  principles  to  directly  address  handling  problem 
content.  First,  the  problem  should  be  aligned  with  the  course  learning  objectives.  The 
course  learning  objectives  should  both  limit  and  define  the  content  of  the  problem.  In 
addition  to  being  aligned  with  course  objectives,  each  problem  should  be  of  scope  proper 
to  the  learning  objectives.  As  an  Anny  example,  a  problem  on  “creative  writing”  would 
be  much  beyond  the  scope  of  an  intended  training  objective  to  write  acceptable  and 
effective  officer  evaluation  reports. 

Outcome  measurement  and  assessmen  t.  Associated  with  the  issue  of  problem 
content  is  the  issue  of  assessing  learners’  outcomes  at  the  conclusion  of  an  instance  of 
PCI.  As  indicated  previously  in  the  discussion  of  transfer  of  learning  (Bransford  and 
Schwartz,  1999),  assessing  the  effectiveness  of  a  learning  experience  depends  very  much 
upon  that  learning  experience’s  objective.  Depending  on  whether  the  objective  of  the 
experience  involves  (a)  learning  factual  knowledge,  (b)  learning  principles  that  link 
together  elements  of  factual  knowledge,  or  (c)  learning  procedures  and  conditions  for 
employing  the  procedures  (cf.  Sugrue,  1995;  Gijbels  et  al.,  2005),  what  is  measured  and 
assessed  should  be  appropriate  to  the  objective.  For  example,  if  the  objective  were  to 
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learn  effective  tactical  machine  gun  employment,  then  an  assessment  of  the  learner’s 
ability  to  load,  fire,  and  reduce  stoppage  on  an  M240B  machine  gun  would  not  be 
directly  appropriate  to  the  objective. 

Gijbels  et  al.  (2005)  and  Sugrue  (2005),  for  each  of  the  three  levels  given 
immediately  above,  suggest  guidelines  for  developing  multiple  choice,  open-ended,  and 
hands-on  based  assessments,  tailored  to  the  level  of  the  knowledge  structure  that  the 
learner  should  have  constructed.  Table  10  contains  an  abbreviated  sample  of  these 
guidelines. 

Table  10 

Example  Measurements  for  Assessment  Method  Formats  for  Knowledge  Levels  from 
Gijbels  et  al.  (2005)  and  Sugrue  (1995) 


Fonnat 

Knowledge  Level 

Multiple  Choice 

Open-Ended 

Hands-On 

Factual  Knowledge 
Principles 

Procedures 

Select  examples 
Select  similar 
problems 

Select  best 
procedure  to  effect  a 
desired  change 

Generate  examples 
Generate  predictions 

Generate  a 
procedure 

Select  live  examples 
Explain  solution 

Perform/  explain 
how  to  perform  a 
procedure 

PCI  instructor  issues.  Consideration  of  constructivist  instructional  issues  such  as 
scaffolding,  problem  generation,  and  learning  assessment  leads  to  consideration  of  the 
role  and  responsibilities  of  the  instructor,  especially  with  respect  to  the  Anny  instructor, 
in  PCI.  As  indicated  previously,  in  a  PCI  enviromnent,  the  instructor’s  role  is  not  so 
much  that  of  a  “broadcaster  of  knowledge,”  but  that  of  a  facilitator.  Blumenfeld  et  al. 
(1991)  note  that  the  PCI  instructor  must  be  expert  in  the  subject  domain,  must  be 
motivated  to  engage  learners  at  a  personal  level,  and  must  be  willing  to  share  with  the 
learner  responsibility  for  the  direction  that  learning  may  take.  These  characteristics  are 
manifested  in  the  PCI  instructor’s  perfonnance.  Ertmer  et  al.  (2009),  after  a  survey  of 
the  literature  touching  PCI  instructors’  perfonnance  and  experiences,  and  after  observing 
five  middle  school  PCI  instructors  over  the  course  of  a  year,  concluded  that  the  effective 
PCI  instructor  is  able  to 

•  Develop  good  problems  and  questions 

•  Anticipate  learners’  questions  and  learning  needs 

•  Deal  with  the  complexity  inherent  in  real-world  problems 

•  Make  appropriate  resources  available 

•  Manage  small  groups 

•  Let  learners  be  responsible  for  their  own  learning 

•  Integrate  learners  with  different  capabilities  into  PCI 
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However,  Dahlgren  et  al.  (1998)  found  that  some  established,  experienced 
teachers,  when  transitioning  to  PCI,  tend  to  retain  what  was  termed  a  “directive”  (as 
opposed  to  a  “supportive”)  role.  Brush  and  Saye  (2006)  further  investigated  this  tendency 
and  found  that  teachers,  even  though  they  are  attempting  to  use  PCI  methods,  may  after 
as  many  as  three  iterations  of  teaching/facilitating  a  course,  continue  to  exhibit 
pedagogical  behaviors  more  appropriate  to  direct  instruction  than  to  PCI.  Dabbagh  et 
al.’s  (2000)  observation  that  “probably  the  most  important  yet  difficult  aspect  of 
problem-based  learning  is  the  role  of  the  instructor  (p  65)”  continues  to  obtain. 

Relative  efficiencies.  Given  the  practical  concerns  associated  with  acceptance  and 
implementation  of  PCI  approaches  to  learning,  from  the  point  of  view  of  Army 
institutional  training,  there  needs  be  some  assurance  of  return  on  investing  effort  in 
developing  PCI.  That  is,  if  the  Army  is  to  commit  to  PCI,  then  there  must  be  some 
indication  that  PCI  can  be  more  effective  than  current  instruction.  Gijbels  et  al.  in  2005 
and,  later,  Walker  and  Leary  in  2009  conducted  meta-analyses  of  studies  that  had  directly 
compared  differences  in  learning  outcome  between  PCI-based  instruction  and  non-PCI 
instruction  (e.g.,  lecture).  Using  Sugrue’s  (1995)  three  knowledge  levels  (see  Table  10 
above)  as  categories  of  instruction,  both  meta-analyses  found  no  difference  between  PCI 
and  lecture  for  “factual  knowledge”  but  a  PCI  advantage  for  both  “principles”  and 
“procedures.”  Note  that  this  result  is  a  comparison  of  PCI  with  lecture;  it  does  not 
necessarily  compare  PCI  with  any  sort  of  optimized  direct  instruction. 

Instructor  Issues 

Army  instructor  preparation.  TRADOC  Regulation  350-70  (In  preparation  - 
b)  outlines  the  Anny  institutional  instructor  training  program.  Although  there  are 
exceptions,  the  most  usual  sequence  is  for  personnel  assigned  to  an  instructor  billet  to  (a) 
complete  the  Army  Basic  Instructor  Course  (ABIC),  (b)  demonstrate  mastery  of  the 
knowledge  domain  to  be  taught,  and  (c)  demonstrate  effectiveness  in  teaching  the 
knowledge  domain.  If  small  group  instruction  is  to  be  employed,  the  instructor  must  also 
complete  the  Army  Small  Group  Instructor  Training  Course  (SGITC).  The  courses’ 
basic  content  (Table  1  la  and  1  lb)  is  centrally  managed  by  TRADOC.  Both  ABIC  and 
SGITC  present  guidelines  and  techniques  under  the  information  processing  model  of 
instruction,  with  SGITC  presenting  additional  guidelines  and  techniques  specific  to  the 
small  group  instructional  environment,  but  neither  course  presents  PCI-related  material. 
Both  courses  are  typically  conducted  at  the  proponent  schools  and  are  conducted  by 
schoolhouse  cadre.  All  new  institutional  instructors,  military,  civilian,  or  contractor, 
typically  complete  the  standard  instructor  preparation  sequence.  Individual  schools  may 
supplement  the  sequence  with  additional  training,  such  as  synchronous/asynchronous  dL 
instructor  training  or  video  teletraining  instructor  training.  A  small  number,  for  example, 
the  U.S.  Army  Command  and  General  Staff  College’s  Faculty  and  Staff  Division  (U.S. 
Army  Combined  Anns  Center,  2009),  provide  alternative  preparation  for  their 
instructors. 
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Table  11. 

High  Level  Characterization  of  Army  Basic  Instructor  and  Small  Group  Instructor 
Training  Courses  ’  Content 


Lesson  Title 

Content 

a.  Army  Basic  Instructor  Course  (ABIC) 

Course  Introduction 

Course  requirements  and  criteria 

Overview  of  Army  Performance 
Improvement 

API  overview,  5  phases  of  API, 
information  about  SAT 

Prepare  for  Instruction 

Student  prepares  selected  topic  for 
delivery  to  peers  and  course  instructor 

Evaluate  Instruction 

Evaluation  techniques;  using  assessment 
to  improve  instruction 

Deliver  (Implement)  Instruction 

Three  student  presentations:  Conference 
method,  Demo/Practical  Exercise  method, 
and  combination  of  the  two 

b.  Small  Group  Instructor  Training  Course  (SGITC) 

Orientation 

Welcome  and  administrative  data. 

Roles,  Responsibilities, and  Definitions 

A  review  of  SGI  terminology. 

Group  Development 

Group  and  individual  dynamics. 

Experiential  Learning  Cycle 

Adult  learning  theory  and  the  ELC. 

Intervention 

Intervention  types,  strategies,  and  pitfalls. 

Leaderless  Discussions 

Brainstonn,  Buzz  Session,  and  Topic 
Discussion. 

Conference 

Directed  large  group  meeting. 

Role  Playing 

Human  interaction  and  discussion. 

Committee  Problem  Solving 

Group  problem  solving. 

Case  Studies 

Case  study  applications. 

Student  Presentations 

Final  exams  for  students. 

Summary 

Course  wrap-up. 
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Army  instructor  characterization.  Typically,  military  instructors  have  no 
formal  training  as  instructor  except  for  ABIC,  SGITC,  and  any  additional  training 
specified  by  the  proponent  school.  In  general,  military  instructors  will  have  had  prior 
experience  in  the  knowledge  domain  to  be  taught.  Civilian  instructors  are  typically  in 
Career  Program  32  (United  States  Anny,  2001),  on  a  professional  career  track  involving 
training  delivery  or  training  support.  They  may  or  may  not  initially  have  expertise  in  the 
knowledge  domain  to  be  taught.  Contractor  instructors’  qualifications  and  experience  are 
specified  by  the  contract  under  which  they  are  hired.  In  most  cases,  contractor  instructors 
have  special  expertise  in  the  knowledge  domain  to  be  taught. 

Military  instructors  are  typically  in  an  instructor  position  from  two  to  three  years 
before  being  re-assigned  to  some  follow-on  billet.  Civilian  and  contractor  instructors  will 
typically  remain  in  an  instructor  position  for  an  indeterminate  length  of  time.  In  general, 
usually  as  a  matter  of  policy,  a  given  instructor  billet  is  not  interchangeable  amongst 
military,  civilian,  and  contractor  instructors.  For  example,  for  the  various  officer 
branches’  captains  career  courses,  nearly  all  small  group  instructors  are  military 
personnel. 

Although  there  is  variation  among  the  Army  schoolhouses,  most  have  drastically 
reduced  their  training  development  staff.  The  result  of  this  reduction  is  that  in  some 
cases  instructors  have  been  obliged  to  assume  some  of  the  training  development  functions 
for  some  courses. 
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Discussion  and  Recommendations 


To  re-iterate,  the  overall  intent  of  this  paper  has  been  twofold:  to  survey  the 
status  of  Army  institutional  training,  especially  with  respect  to  dL,  and  to  survey  current 
training  research  and  technology  as  applicable  to  the  Army’s  institutional  training 
environment.  Although  various  “gaps”  in  institutional  training  are  pointed  out  in  the 
preceding  sections,  in  this  final  section,  there  is  a  high-level  “sweep  up”  of  all  the 
significant  findings  of  the  paper  and  a  general  listing  and  discussion  of  issues.  The  listing 
has  two  general  categories:  policy-related  findings  and  research-related  findings. 
Although  the  distinction  is  sometimes  blurred,  the  policy-related  items  are  those  issues 
that  appear  to  have  direct  solutions  that  need  little  or  no  further  analysis  or  that  are 
outside  ARI’s  purview.  The  research-related  items  are  those  issues  that  appear  to  have  no 
directly  available  solution,  and  need  additional  analysis  before  a  solution  can  be 
considered. 

Policy  Issues 

These  overall  issues  are  of  a  general  nature  and  applicable  across  Anny 
institutional  training. 

Soldier-related  issues.  Currently  “task-condition-standard”  is  both  a  training 
and  a  Soldier  mantra.  Although  per  se  this  is  an  admirable  training  goal,  it  is  an  effective 
goal  only  to  the  extent  that  the  tasks,  conditions,  and  standards  encountered  in  the  field 
are  similar  to  the  task,  condition,  and  standard  encountered  in  the  institutional  setting.  A 
reliance  on  a  small  set  of  tasks,  conditions,  and  standards  in  the  classroom  reflects  the 
mindset  of  training  to  meet  course  exit  criteria  and  is  somewhat  incomplete.  As  Barnett 
and  Ceci  (2005)  aver,  “no  one  cares  about  learning  if  it  stops  at  the  schoolhouse  door”  (p. 
295).  Soldiers’  learning  orientation  must  be  toward  subject  matter  mastery  and  not  just 
performance  to  classroom  standard.  There  needs  to  be,  at  a  high  level,  a  change  from 
measuring  Soldiers’  end-of-course  performance  to  measuring  the  transfer  of  learning  in 
the  field  that  results  from  a  course  of  instruction. 

Course  relevance.  If  the  Army  is  to  maintain  a  life  long  learning  environment, 
then  the  Anny  should  continue  to  pursue  a  policy  of  insuring  that  course  content  is 
relevant  and  up  to  date.  To  be  relevant,  training  must  be  valid.  Course  validation  needs 
to  not  only  emphasize  content  validation  as  outlined  in  TRADOC  Pamphlet  350-70-10 
(U.S.  Army  Training  and  Doctrine  Command,  2004)  but  also  emphasize  external 
validation  (cf.  Straus  et  al.,  2006). 

From  the  Army  point  of  view,  another  aspect  of  relevance  is  currency: 
institutional  courses,  especially  those  applicable  to  the  current  operational  environment, 
must  tighten  their  update  cycle.  This  will  involve  both  process  and  product  changes. 
Efforts  must  be  made  to  make  more  efficient  the  process  by  which  changes  are  fed  back 
into  to  the  course  development  function  and  the  process  by  which  courses  are  updated. 
Courses,  especially  dL  courses  and  dL  course  components  of  blended  learning,  must  be 
designed  with  specific  consideration  of  updates  and  modifications  kept  in  mind.  That  is, 
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there  should  be  a  conscious  change  of  mindset  within  the  course  development  function  to 
design  not  to  accommodate  static  course  content,  but  to  accommodate  a  dynamic  course 
content  that  changes,  sometimes  very  rapidly,  over  time. 

Course  availability.  The  Army  should  continue  to  pursue  a  policy  of  making 
the  right  courses  available  to  Soldiers  at  the  right  time.  The  Army  should  continue  to 
pursue  a  policy  of  aligning  course  offerings  within  the  ARFORGEN  cycle.  In  line  with 
this,  the  Anny  also  should  visit  the  current  policies  of  managing  residential  courses  at  the 
course  level.  With  fixed  residential  course  start/end  dates,  any  benefits  to  be  had  from 
developing  individually  self-paced  courses  are  not  being  exploited.  With  this  in  mind,  a 
trade-off  between  the  training  effectiveness  gained  via  self-paced  courses  versus  the 
increased  administrative  overhead  of  managing  self-paced  courses  should  be  conducted, 
with  the  results  to  govern  whether  the  Anny  invests  significantly  in  developing 
institutional  self-paced  courses. 

Transfer  related  issues.  Not  only  must  a  Soldier’s  training  be  oriented  beyond 
the  end  of  the  course,  but  also  the  work  environment  to  which  the  Soldier  returns  must 
support  the  Soldier’s  new  learning.  Leaders  must  recognize  that  to  maximize  the  return 
on  training  investment,  the  Soldier  must  be  allowed  the  opportunity  and  resources  to 
perform  the  newly  learned  skills.  To  the  extent  that  the  Soldier’s  leadership  and  peers 
support  and  value  the  newly  learned  skills,  opportunity  for  transfer  to  the  operational 
environment  will  be  optimized.  Once  emphasis  shifts  from  end-of-course  assessment  to 
assessment  on  the  job,  there  will  be  a  concomitant  need  to  broaden  the  focus  of 
assessment  to  include  not  only  Soldiers  but  also  training  systems.  This  broadening  is  also 
discussed  as  a  research  issue  below. 

dL-specific  issues.  The  Anny  has  continuing  problems  with  acceptance  of  dL, 
by  leaders,  by  developers  and  instructors,  and  by  students.  Many  Army  senior  leaders 
have  yet  to  firmly  endorse  dL.  Developers  and  instructors  are  conflicted  by  policies  that 
are  perceived  to  undennine  job  security  or  to  mandate  conversion  to  dL  at  a 
programmatic  level  rather  than  at  a  learning  requirements  level.  Students  form  a  vision 
of  dL  modules  as  non-imaginative  “page  turners”  that  are  unwieldy  to  access  and  many 
times  out  of  date. 

Students’  visions  of  dL  reflect  the  Army’s  continuing  problems  with  its  dL 
infrastructure.  End  users  continue  to  report  annoying  to  unacceptable  delays  between 
user  input  and  system  response  while  they  attempt  to  complete  dL  courses.  The  Army 
has  little  control  over  the  delay  attributable  to  the  user’s  own  platform  (usually  a  PC), 
some  control  over  the  transport  (network)  delay,  and  most  control  over  delay  due  to  the 
servers’  processing  time.  The  fact  that  course  developers  have  come  to  consciously  avoid 
incorporating  potentially  effective,  but  processing  intensive,  features  in  dL  modules  due 
to  anticipated  response  time  issues  speaks  to  the  persistence  of  this  problem. 

Our  finding  of  students’  having  impressions  of  dL  as  having  uneven  quality  are 
supported  by  Straus,  Shanley,  Bums,  Waite,  and  Crowley  (2009)  who  in  a  TRADOC- 
sponsored  study  concluded  that  a  good  proportion  of  Army  dL  up  until  now  has  suffered 
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from  lack  of  a  systematic  quality  assessment  process.  The  Straus  et  al.  report 
recommends  TRADOC  bring  to  bear  an  emphasis  on  assessing  technical  criteria,  such  as 
accessibility  and  usability;  production  quality  criteria,  such  as  legibility  of  material;  and 
pedagogical  criteria,  such  as  characteristics  of  instruction  and  student  feedback. 

Finally,  the  Army  should  actively  pursue  more  dL  collaboration  with  other 
agencies.  While  the  majority  of  dL  projects’  content  is  likely  to  be  Army-specific,  there 
are  economies  to  be  investigated  with  respect  to  dL  for  common  content  and,  more 
importantly,  with  respect  to  dL  process  commonality  and  lessons  learned. 

Research  Issues 

These  significant  research  issues  are  re-iterated  for  specific  consideration  for 
inclusion  in  any  program  of  research  addressing  effectiveness  of  Anny  institutional 
training.  Their  scope  ranges  from  basic  research  to  advanced  development,  and  some  can 
be  quickly  addressed  while  others  may  require  more  time. 

The  Army  learner.  The  Army  has  for  some  time  espoused  the  concept  of  each 
Soldier  being  a  self-directed  life-long  learner.  While  there  are  increasingly  more 
opportunities  for  Soldiers  to  learn,  there  are  as  yet  no  specified  meta-principles  for 
helping  Soldiers  learn  to  learn.  In  particular,  there  is  a  need  for  Soldiers  to  assume  more 
personal  responsibility  for  learning  the  knowledge  and  skills  necessary  to  effectively 
execute  their  chosen  career  trajectory.  There  is  a  need  for  research  oriented  toward 
identifying  those  leamable  skills  that  enable  the  Soldier  to  be  a  life-long  learner  and 
toward  determining  the  extent  to  which  those  skills  should  change  across  a  Soldier’s  full 
career.  Some  work  in  this  area  is  being  sponsored  by  TRADOC ’s  recently  constituted 
Institute  for  NCO  Professional  Development  (Kim,  2009). 

Loosely  associated  with  the  problem  of  learning  to  be  a  life-long  learner  is  that  of 
the  issue  of  the  “average”  level  of  public  school  education  attained  by  a  Soldier  before 
embarking  upon  life-long  learning.  Research  is  needed  to  determine  if  success  as  a  life¬ 
long  learner  is  significantly  correlated  with  level  of  public  education  attained  prior  to 
entering  service  and,  if  so,  what  could  be  done  to  modify  training  to  match  different  year- 
groups’  level  of  public  education. 

Training  for  transfer.  If  the  Anny  is  to  move  from  evaluating  instruction  by 
depending  on  end  of  course  assessments  to  evaluating  instruction  by  depending  on 
subsequent  changes  in  Soldiers’  operational  perfonnance,  there  must  first  be 
implemented  some  process,  similar  to  Holton’s  (2008)  LTSI,  that  views  transfer  from  a 
system  perspective  (Holton,  et  al.,  2000).  Similar  to  components  of  the  LTSI  model, 
there  is  a  need  to  catalog  and  to  assess  the  roles  of  components  of  the  Army  training  and 
operational  environments  on  transfer  of  training  to  operational  performance. 

In  parallel  with  this  investigation,  there  needs  to  be  research  in  the  area  of 
assessment  of  transfer.  Instead  of  treating  transfer  as  an  all-or-none  effect,  researchers 
need  to  be  able  to  assess  it  as  a  graduated  effect  (e.g.,  Bransford  et  al.,  2000)  and  also  to 
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assess  it  dynamically  (e.g.,  Schwartz  and  Martin,  2004)  in  terms  of  Soldiers’  preparation 
for  new  learning  and  for  learning  in  new  situations. 

Probably  most  importantly,  there  is  a  need  to  investigate  pedagogical  methods  for 
improving  transfer  to  the  operational  environment.  In  fact,  since  the  aim  of  Army 
training  and  education  in  most  cases  is  to  affect  performance  in  the  operational 
environment,  the  case  can  be  made  that  any  investigation  of  pedagogical  method  within 
the  Army  institutional  training  be  oriented  toward  transfer  to  the  operational 
environment.  The  pedagogical  issues  outlined  in  the  next  section,  when  addressed, 
should  be  researched  with  respect  to  transfer.  When  considered  in  this  light,  it  becomes 
clear  that  the  effect  of  a  pedagogical  intervention  must  be  investigated  in  terms  of  not 
just  course  process  and  content,  but  also  the  post-course  factors  that  bear  on  Soldiers’ 
operational  perfonnance. 

Pedagogical  issues.  Pedagogical  issues  revolve  around  Mayer’s  (1996)  three 
metaphors  of  behaviorism,  information  processing,  and  constructivism.  One  of  the  basic 
issues  is  that  of  optimally  matching  metaphor  to  the  domain  the  student  is  to  master. 
While  there  is  no  question  that  all  three  models  of  instruction  are  effective,  there  are  still 
only  general  guidelines  as  to  which  one  will  be  most  effective  for  which  kinds  of 
instructional  goals  and  also  the  extent  to  which  each  model  should  be  used  and  at  what 
point  during  the  course  of  instruction  for  a  knowledge  domain. 

Aside  from  research  addressing  the  relative  merits  of  the  three,  there  are  also 
needs  for  research  directed  specifically  at  the  information  processing  metaphor  and  the 
constructivism  metaphor. 

Information  processing  related  issues.  Because  processing  in  this  metaphor  is 
based  in  the  hypothetical  structure  of  short  term  memory,  the  majority  of  research  issues 
revolve  around  short  tenn  memory. 

Short  term  memory  capacity  issues.  Since  short  term  memory  is  hypothesized  as 
the  bottle-neck  for  learning,  its  processing  capacity  is  of  critical  importance  to  the 
information  processing  metaphor.  There  are  two  related  main  research  issues  associated 
with  capacity.  First,  of  course,  is  the  number  of  items  that  can  be  maintained  in  short 
tenn  memory,  and  second  is  the  nature  of  short  term  memory  items  and  whether  short 
tenn  memory  capacity  changes  as  a  function  of  the  nature  of  the  items  held  in  memory. 
Theoretically  based  admonitions  to  avoid  exceeding  short  tenn  memory  capacity  during 
training  cannot  be  implemented  in  instruction  absent  some  useful  non-specific 
characterization  of  what  constitutes  a  short  term  memory  item. 

Cognitive  load  issues.  Deriving  from  the  issue  of  short  term  memory  capacity  is 
the  issue  of  cognitive  load  -  if  short  term  memory  has  some  limiting  capacity,  then  for 
learning  to  effectively  occur,  the  presented  intrinsic,  gennane,  and  extraneous  loads  must 
be  quantifiable.  This  quantification  includes  not  only  the  number  of  intrinsic,  gennane, 
and  extraneous  items  presented,  but  also  the  rate  at  which  new  items  can  be  presented  as 
older  items’  information  is  assimilated  (in  the  case  of  intrinsic  and  gennane  elements)  or 
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disregarded  (in  the  case  of  extraneous  elements)  and  the  items  themselves  dropped  from 
short  term  memory.  Research  is  needed  to  characterize  the  rate  at  which  items  of  various 
types  may  be  presented  the  learner  without  overload  and  thus  optimizing  learning  rate. 

PCI  related  issues.  As  PCI  methods  are  incorporated  into  institutional  training, 
issues,  some  specific  to  Army  training,  arise  that  should  be  addressed. 

Problem  generation  issues.  For  a  course  presented  using  PCI,  the  basic  element 
from  which  instruction  derives  is  the  learning  problem.  Given  that  the  instructor’s  role  in 
PCI  is  more  supportive  than  directive,  problems  must  be  carefully  crafted  to  insure  that 
the  learners’  trajectory  through  the  solution  process  will  treat  all  the  intended  course 
objectives.  In  the  Army  environment,  an  investigation  of  problem  generation  will 
involve  not  only  training  developers,  but  also  the  instructors  who  will  be  providing  the 
scaffolding  to  shape  and  guide  the  problem  through  to  its  solution  by  the  learners. 

Scaffolding  issues.  Scaffolding  can  be  compared  to  germane  load  in  the 
information  processing  metaphor.  As  initially  introduced  to  the  field  of  education, 
scaffolding  referred  to  focused,  individualized  support  for  a  learner,  ostensibly  aimed  at 
an  individual’s  zone  of  proximal  development.  However,  when  the  learning  environment 
expands  to  the  classroom,  individualization  of  scaffolding  becomes  more  and  more  of  a 
challenge  to  the  instructor:  how  much  scaffolding  can  be  provided  on  an  individual 
basis;  when  does  providing  individualized  scaffolding  begin  to  interfere  with  classroom 
flow;  and  what  are  the  pro’s  and  con’s  of  providing  scaffolding  at  the  class  level  instead 
of  at  the  individual  level.  Some  nominal  work  in  this  area  has  been  done.  Saye  and 
Brush  (2002)  make  the  distinction  between  hard  scaffolding  that  is  pre-planned  for 
delivery  at  distinct  points  in  instruction  and  soft  scaffolding  that  is  delivered  on  an  ad 
hoc,  as  needed  basis),  but  little  empirical  work  has  addressed  issues  of  scaffolding 
efficiency. 

There  is  a  need  to  investigate  relationships  amongst  subject  matter  complexity, 
scaffolding,  and  group  size.  For  less  complex  subject  matter,  hard  scaffolding  and  larger 
group  sizes  may  be  effective  and  efficient.  On  the  other  hand,  for  more  complex  subject 
matter,  soft  scaffolding  and  smaller  group  sizes  may  be  required  to  attain  training 
effectiveness. 

Associated  with  scaffolding  is  the  question  of  the  number  of  small  groups  an 
instructor  can  divide  a  given  class  into  as  he  or  she  progresses  through  a  course.  If 
indeed  it  is  the  case  for  PCI  that  more  complex  subjects  require  smaller  groups  and  more 
soft  scaffolding,  then  it  may  be  the  case  that  instructors  must  invest  relatively  more  and 
more  instructional  oversight  per  group  as  group  size  decreases. 

Training  for  PCI  instructors.  The  Army’s  current  process  for  qualifying 
instructors  does  not  prepare  instructors  in  the  use  of  PCI  methods.  In  view  of  the  up¬ 
coming  introduction  of  PCI  methods  in  Army  training  and  education  (U.S.  Army 
Training  and  Doctrine  Command,  In  Preparation-b),  there  is  a  need  to  survey  methods  for 
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preparing  instructors  to  use  PCI  and  to  match  and  validate  those  methods  in  the  Anny 
institutional  training  environment. 


Overall,  this  survey  finds  that  Army  institutional  training  is  in  reasonably  good 
condition.  Even  though  there  are  technological  constraints  on  training  delivery  and 
management,  institutional  training  appears  to  be  making  progress  toward  implementing 
blended  learning  solutions,  where  before  instruction  was  primarily  lecture-style.  There  is 
movement  toward  incorporating  advanced  pedagogical  techniques  into  instruction,  but 
the  gaps  between  the  Army’s  training  needs  and  what  is  operational  usable  from  those 
techniques  require  additional  investigation. 
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Appendix  A 

dL  Institutional  Training  Interview  Questions 


Course  Development 

1.  For  what  courses  have  you  developed  dL? 

2.  What  is  the  primary  teaching  objective  of  the  dL  courses  you  have  developed  to  date? 

a.  To  teach  declarative  knowledge? 

b.  To  teach  process  knowledge  (how  something  works)? 

c.  To  teach  procedures  (how  someone  does  something)? 

d.  To  teach  causal  principles  (cause  and  effect)? 

3.  Do  you  follow  any  specific  learning  theory,  model,  or  principles  in  developing  your  training 
courses?  (e.g.,  GEL,  SRS) 

a.  If  yes,  please  describe. 

b.  If  yes,  does  the  course  content/objectives  (e.g.,  knowledge  vs  application;  skill  transfer 
vs  procedural  task  that  does  not  change;  skills  that  require  job  aids  vs  skills  that  do  not) 
determine  the  choice  of  a  learning  model  or  principles? 

c.  If  no,  then  how  do  course  content/objectives  drive  your  development  strategy? 

4.  What  is  the  most  important  guiding  factor  in  your  course  development? 

a.  Mandates  from  TRADOC? 

b.  Learning  theory  or  model? 

c.  Training  objectives? 

d.  Training  resources? 

e.  Operational  needs? 

f.  Other? 

5.  To  what  extent  have  user/Soldier-SMEs  been  involved  in  course  development? 

Course  Evaluation 

6.  To  what  extent  have  you  incorporated  some  type  of  formative  evaluation  strategy  to  assist  in 
the  development  of  your  instructional  training  programs? 

a.  If  so,  describe  the  strategy  that  you  used. 

b.  What  do  you  do  to  maximize  the  effectiveness  of  conducting  these  evaluations? 

7.  How  do  you  evaluate  whether  your  courses  meet  stated  training  objectives? 

a.  What  types  of  information  do  you  typically  collect  as  part  of  your  course  evaluations? 

i.  Student  reactions? 

ii.  Self-assessment? 

iii.  Acquired  knowledge? 

iv.  Behavioral  measures? 

1 .  To  what  extent  do  you  attempt  to  collect  follow-up  performance  data  from 
students  in  the  work  place? 

2.  What  issues  have  you  encountered  in  trying  to  collect  this  type  of 
information,  e.g.,  transfer  of  training? 
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8.  To  what  extent  do  you  measure  results ?  (Results  =  productivity  gain  by  the  unit  or 
organization). 

a.  How  difficult  is  it  to  collect  “bottom  line”  data? 

b.  What  are  the  major  issues  you  have  faced  in  trying  to  collect  bottom-line  data? 

i.  Identifying  measures  that  can  be  interpreted  clearly? 

ii.  Long-term  tracking  of  students/unit? 

iii.  Resources? 

9.  What  type  of  evaluation  strategies  (experimental/quasi-experimental)  have  you  used  to 
determine  or  compare  the  effectiveness  of  a  particular  learning  strategy/technique  or  training 
media  feature? 

i.  For  example: 

1 .  one-group  posttest 

2.  two-group  posttest  only  with  nonequivalent  comparison  group 

3.  two-group  pretest-posttest  with  nonequivalent  comparison  group  multiple 
posttests 

ii.  What  problems  have  you  encountered  in  conducting  this  type  of 
experimental/quasi-experimental  research? 

10.  If  you  were  asked  to  develop  a  set  of  evaluation  measures  for  a  training 
course,  what  type  of  information  would  you  most  like  to  have? 

a.  What  problems  have  you  encountered  or  can  you  expect  others  to  encounter  in 
collecting  this  information? 

1 1 .  How  do  students  perceive  the  value  of  the  course? 

a.  How  favorably  do  students  respond  to  the  training  content  and  delivery? 

i.  Relevance  of  the  training  to  the  job? 

ii.  The  training  technologies  used,  the  quality  of  audio  and  video,  the  overall  training 
environment,  access  to  live  instructors? 

12.  What  issues  (e.g.,  developing  valid  measures,  administrating  tests,  updating 
tests)  have  you  faced  in  developing  evaluation  instruments  of  which  course 
developers  should  be  aware? 

13.  What  issues  have  you  encountered  with  students  interfacing  with  the  dL  content ? 

a.  What  navigation  issues  emerged? 

b.  How  user-friendly  was  the  configuration? 

14.  What  features  or  aspects  of  dL  do  students  generally  like? 

15.  What  features  or  aspects  of  dL  do  students  generally  dislike? 

Lessons  Learned 

16.  Based  on  your  experience,  what  valuable  lessons  have  you  learned  about  the  following: 

a.  Development 

b.  Implementation 

c.  Maintenance/management 

d.  Evaluation 

e.  Resource  issues 
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